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COMPREHENSIVE SERVICE INTEGRATION PROGRAMS 
FOR AT-RISK YOUTH: FINAL REPORT 

EXECUTIVE SUMMARY 



Estimates suggest that as many as half of today’s youth run a moderate to high 
risk of experiencing school failures or participating in early sexual activity, alcohol 
and drug use, and criminal behaviors. This report examines programs targeting at- 
risk younger adolescents, aged 10 to 15. 

The results of many years of program impact evaluations demonstrate that 
single-focus programs targeting at-risk adolescents may not be the most effective way 
to help youth. Increasing attention is being paid to programs capable of dealing with 
the whole child, including the child’s parents and neighborhood. Thus, a major focus 
of this report is to learn more about the ability of programs to provide more 
comprehensive services to youth through service integration. It also examines the 
barriers and successes programs encounter while attempting to do so. 



Project Objectives 

The objectives of this project are to: 

• Document how comprehensive, integrated services are delivered to at-risk 
youth between the ages of 10 and 15. 

• Identify effective methods of providing comprehensive, integrated services for 
this population. 

• Identify barriers to providing comprehensive services, and means of facilitating 
service integration for at-risk youth. 

• Examine the role of Federal, state, and local government and the nonprofit 
sector in impeding or facilitating service integration for at-risk youth. 

• Examine the extent to which simple lack of services, or insufficient service 
capacity, is implicated as a barrier, in comparison with eligibility, regulatory, 
jurisdictional, and other factors. 

• Identify issues for further research on the provision of comprehensive, 
integrated services for at-risk youth. 



Methods 

To meet these objectives we reviewed the literature concerning the meaning of 
risk, the prevalence of risk behaviors among youth, and the successes and limitations 
of traditional programs serving youth. We conducted a literature review and 
examined issues related to evaluating programs for at-risk youth, and then conducted 
site visits to nine programs in six locations. 
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Report Outline 

Chapter 1 introduces the study and its objectives. Chapter 2 contains a review of 
the literature on youth -at risk. It examines definitions and prevalence of risk, 
describes current approaches to service delivery, explores the definition of and 
motivation behind service integration (SI), and examines the barriers programs face to 
implementing SI. Chapter 3 delineates the evaluation issues concerning programs 
serving youth, at risk. Chapter 4 introduces the objectives of the site visits and 
describes the procedures used. Chapter 5 describes each of the nine programs^, 
visited. Chapter 6 identifies and discusses issues that cut across programs. Chapter 
7 presents a summary of the project and implications of the findings. 



Risk Definitions, Risk Prevalence, and Service Integration Issues (Chapter 2) 

Some of the key issues discussed in detail in Chapter 2 include the definition of 
risk used during the project, the difficulties of measuring prevalence of risk, the 
limitations of traditional services that focus on single problems rather than on 
meeting the needs of the whole individual, our definition of service integration (SI), the 
barriers to service integration, and the crucial steps that should be taken to plan and 
implement SI. 

The Meaning of Risk 

The conceptual framework for defining risk was developed for this report and 
consists of four components: 

• Risk antecedents: Those environmental forces that have a negative impact on 
the developing individual by producing an increased vulnerability to future 
problems in the family, school, or community. Based on our review of the 
literature, there appear to be three critical risk antecedents for early adolescents: 
poverty, neighborhood environment, and family environment. 

• Risk markers: These are visible indicators of behavior, in public records. 

Previous research suggests a consistent relationship between these behaviors and 
risk antecedents, and a well-defined link with increased vulnerability and the 
onset of potentially negative behavior. We have' selected two indicators that are 
consistently identified as markers for all problejn behaviors of adolescence: poor 
school performance and involvement with child protective services, including out- 
of-home placement in the foster care system. These two have particular policy 
relevance because they can be observed in the records of public systems, and 
allow program planners to target the youth at greatest risk. 

• Problem behaviors: These are defined as activities that have the potential to hurt 
youth, the community, or both. Research has identified these behaviors as those 
most likely to occur in youth who, earlier, displayed risk markers, or who were 
living under risk antecedent conditions. We have chosen those behaviors that 
have most consistently been identified in the literature as signalling potentially 
more serious consequences for youth in the future, including: early initiation and 
practice of sexual behavior, truancy or absenting from school, running away from 



home (or from an out-of-home placement), early use of tobacco, alcohol, and 
other drugs, and associating with delinquent peers. 

• Risk outcomes: These are clearly injurious conditions that have negative 
consequences for a youth’s future development as a responsible, self-sufficient 
adult. The risk outcomes of primary concern include teenage pregnancy/ 
parenthood, homelessness, involvement in prostitution, alcoholism and/or drug 
abuse, delinquency and criminal behavior, school dropout, AIDS, chlamydia and 
other sexually-transmitted diseases, physical and sexual abuse, and various 
morbidity and mortality conditions (hepatitis, tuberculosis, pneumonia, 
accidents, suicide, homicide). 



Prevalence of Risk 

This report examines the prevalence of the above four components based on the 
secondary literature. Despite the apparent overlap in risk antecedents and markers, 
it is difficult to develop a composite estimate of the degree to which adolescents run a 
high, moderate, or low risk for engaging in problem behaviors or experiencing risk 
outcomes. Using a simple population estimate based on poverty or neighborhood is 
very rough and will overestimate the number of youth who go on to experience risk 
outcomes. The more precision one desires in an estimate of risk, the more difficult 
the task becomes. This is because antecedents and markers are never perfect 
predictors, and the quality of data gets significantly worse as the variables are more 
closely connected to problem behaviors or risk outcomes. Some investigators address 
this problem by taking the presence of the problem behaviors themselves as their 
"risk" indicators. But this approach merely begs the question, since the youths it 
selects as high risk have already done the things one is interested in predicting. A 
better solution is to use several antecedent and marker variables to predict risk: in 
general, the more relevant the variables included, the more precise prediction one 
achieves. 

Limitations of Traditional Services 

Our survey of traditional services for at-risk youth shows that they often address 
only a single risk marker or outcome such as adolescent pregnancy, substance abuse, 
or school failure. This single-problem focus has several limitations. First, such 
programs usually focus on problems (rather than individuals as a whole) and tend to 
offer short-term interventions. Programs that try to solve problems quickly and then 
close the case are not geared toward preventive interventions and often have little 
staying power. Thus, they do not always address the most pressing needs of their 
clients. Second, it is sometimes difficult to get other community agencies to fill in the 
gaps when such single-issue programs cannot meet client needs with their own 
program resources. Given these problems, programs have tried to increase the 
comprehensiveness of their own offerings and use service integration to increase their 
clients’ access to a wide range of services offered by other programs and agencies. 
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Definition of Service Integration 



By "service integration" (SI), we refer to procedures and structures that help 
several service agencies coordinate their efforts to address the full range of service 
needs presented by youth and families in an efficient and holistic manner. It is 
important to understand that, despite the common tendency to refer to 
"comprehensive service integration programs," the terms "comprehensive" and "service 
integration" are not synonymous. Service integration is merely one method of 
obtaining comprehensive service coverage, but SI does not guarantee comprehensive 
service coverage. Furthermore, programs may use different combinations of 
comprehensiveness and SI. For instance, a mental health program may have 
arrangements with other agencies to provide additional services, but these services 
may still be related to mental health rather than to other aspects of client need such 
as housing or education (this is an example of SI without comprehensiveness). Or, a 
program may try to make its own service offerings comprehensive rather than relying 
on outside agencies to fill in service gaps (this is an example of comprehensiveness 
without SI). 

Few existing systems meet all the elements of the SI mode 1 with which this 
inquiry began. Several key elements in this initial idea of SI efforts for at-risk youth 
are: 

• An approach to helping at-risk youth that sees each youth for himself or 
herself, and also sees the youth as part of a family, neighborhood, and 
community that may in turn be influenced to reduce the risk that a youth will 
participate in problem behaviors or experience risk outcomes. 

• A comprehensive, individualized assessment at or near the point of intake, that 
is conducted for each youth and family, to identify the full range of his or her 
individual and family service needs. 

• A coordinated service plan that, based on the needs identified, is 
developed to ensure that all needs are addressed in an efficient fashion 
by the program(s) best suited for the task. 

• Institutionalized interagency linkages that ensure that service referrals 
result in actual service delivery. This may entail an interagency case 
management function, co-location of services at a single site, and/ or 
sharing of other resources among programs. 

• Follow-up on service referrals, to ensure that services are delivered in an 
appropriate manner and that the program coordination structures are 
functioning effectively. 

Barriers to Service Integration 

SI efforts face many barriers, including professional training and orientation, 
administrative procedures, eligibility rules, and the categorical nature of funding. 
Service agency staff are typically trained in rather narrow, specialized traditions such 
as mental health or criminal justice services, and may not feel comfortable dealing 
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with other issues or working within an interagency framework. Bureaucrat c 
procedures often obstruct SI efforts because agencies may insist on following their 
own intake and case processing procedures, and confidentiality requirements may 
limit their ability to share information about clients with an SI team. 

Categorical public and private funding also perpetuates single-issue programs. 

As long as legislatures and funders structure programs to address specific problem 
areas, single-issue programs will continue to have difficulty making their services 
available to populations not specified by their mandate. 

Steps to Planning and Implementing SI 

Some of the crucial steps that should be considered when planning and 
implementing service integration include: 

• Defining Goals and Objectives. Encourage long-term commitment to the 
integration effort by creating an independent interagency advisory group to help 
minimize turf battles and forge a common purpose for service integration 
partners. 

• Identifying the Target Population. Unless the target population is clear, it will 
not be obvious what services should be incorporated into the effort. There is no 
definitive profile of youth or families who need SI. However, prime candidates 
may include families who need help in supportive parenting due to involvement 
in alcohol or drug treatment or with child welfare because of reports of abuse 
or neglect. Equally important are families who have none of these problems 
but who struggle to raise their children with little money and few resources in 
neighborhoods that pose a constant threat to their children’s future. 

• Identifying the Services to Be Offered. A comprehensive approach requires 
considerable variety in the breadth and depth of services available and 
flexibility in service deliveiy. The type of services to be offered should be 
determined on the basis of local needs and resources. 

• Mechanisms for Service Delivery. Services may be coordinated in different 
ways. Clients may have a service agency contact with whom they maintain an 
ongoing, supportive relationship. When this contact person functions more as 
a mentor, counselor, or group worker than as a case manager, this individual 
needs access to someone who can arrange needed services and follow up on 
referrals. 

• locating the Service Site. Integrated services can be delivered through school- 
based or school-linked sites, in community sites, through mobile arrangements, 
or by home visits. The location of an SI effort most likely depends on which 
agency or group has a committed and dynamic person willing to take the lead 
in developing and running the program. The site’s acceptance in the 
community is also an important factor. Debates about the appropriate balance 
of services between on-site and off-site locations center around the relative 
benefits of ease of access versus teaching clients to negotiate the system 
themselves. 
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• Eliminating Administrative Barriers to SI. Agencies participating in SI should 
have institutionalized linkages that establish the mechanisms for sharing 
resources. These mechanisms may include co-locating in a single facility, 
sharing staff financial resources and/or information, and agreeing to provide 
services to referred people. 

• Hiring Staff. Staff should be selected very carefully. Criteria should include 
their: ability to establish trusting, respectful relationships with youth and 
families; ability to span professional boundaries to address clients’ needs; 
willingness to support the SI model; and demonstrated sensitivity to issues of 
racial, ethnic, and gender diversity. 

• Creating Flexible Funding. For SI to work best, funding should be flexible, 
avoiding the rigidities of circumscribed service delivery and the eligibility 
difficulties associated with categorical funding. Federal and state funding 
sources should be redesigned to blend together funds from multiple sources 
that historically have rigid categorical boundaries, to provide adequate and 
coherent funding for service programs that address multiple areas of need. 

One promising approach to increasing SI among already functioning programs 
is using limited new funding to support core SI functions. This effort could be 
matched by diverting some existing funds to support additional SI efforts and 
using other existing funds to support regular service delivery— an approach now 
being used in Kentucky. 

• Designing and Using Evaluations Effectively. Evaluators must have extensive 
early collaboration with program personnel so the measures used are 
meaningful and cooperation with the evaluation is high. Impact information 
should be tied to youth and family outcomes rather than simply services 
delivered. Programs that look good as demonstrations are often diluted upon 
replication, suggesting that evaluation results are not reviewed in enough detail 
to assure that critical aspects of programs actually appear in replication. 

• Institutionalizing Change. A major goal of SI is to change service delivery 
systems in a permanent way. But this often does not happen; changes rarely 
survive the tenure of the key people involved in SI efforts. For SI efforts to 
produce true system change, the interagency linkages and ways of interacting 
must be codified into a new approach to standard operating procedure. 



Evaluation Issues and Lessons Learned (Chapter 3) 

Chapter 3 documents evaluation issues specific to youth-serving programs, to 
service integration efforts, and to the types of sites selected for this study. Issues 
examined include defining the participant and the unit of analysis, measuring client 
risk levels and including them in analyses to understand program impacts, 
documentation of service delivery, non-client outcomes of interest, differentiating the 
impact of SI from that of comprehensiveness, evaluation readiness, identifying 
realistic outcomes to measure, identifying appropriate comparison groups, and 
reducing attrition to follow-up. The chapter discusses: 



• The need to adjust evaluation designs to reflect major elements of program 
activity, including prevention. 

• The importance of assessing client risk levels, documenting service delivery, 
and including appropriate indicators in a multivariate impact analysis. 

• Incorporating measures of the extent of SI, comprehensiveness, program design 
change, and system change as important aspects of evaluation design for the 
types of programs examined here. 

• The importance of using a strong evaluation design (probably quasi- 
experimental), having adequate instrumentation to measure key concepts, and 
obtaining follow-up data from a very high proportion of entiy cohorts and 
comparison group members. 



Some of the lessons learned from years of program evaluations are as follows: 

• Evaluators should be outsiders rather than program staff, but these individuals 
need to take the time to get to know the program and work carefully with 
program staff to develop mutually agreeable arrangements. 

• The impacts that programs care most about, such as youth development or 
leadership training, may be the most difficult to measure adequately. 

• An exclusive focus on outcomes and impacts does not always accurately 
capture the full picture. Quantitative outcomes should be augmented by using 
qualitative and observational methods to learn not just whether a program 
"works," but how it works, under what conditions, and for whom. Not knowing 
these specifics about program-client fit makes it harder to recommend future 
applications of a demonstration program or to translate results into broader 
policy directions. 



Site Visit Objectives and Procedures (Chapter 4) 

The site visits were conducted: 

• To understand the full range of program configurations and options for 10- to 
15-year-olds, including the programs' sense of their mission or purpose. 

• To understand the reasons behind these programs’ choices among certain 
program design alternatives (e.g., whether to emphasize "activities" or 
"services;" whether to concentrate on prevention or on treatment; whether to 
adopt a focus on youth, on youth plus their families, on families in general, or 
on the total neighborhood; whether to strive for comprehensive service delivery). 

• To understand the relationship of these programs to their larger community, 
including both the program’s role in the service delivery network and network 
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of supports for youth, and the program’s role in relation to other community 
institutions such as churches and community centers. 

• To learn what programs believe are the benefits of a more comprehensive range 
of services, and what they believe are the benefits and drawbacks of service 
integration through collaborative arrangements with other agencies. 

• To gain a sense of the readiness and willingness of programs of this type to 
participate in evaluations, and what types of evaluations they might be open to 
(or have already been involved in). 

Site Selection Criteria 

We looked for programs that serve clients between the ages of 10 and 15; conduct 
comprehensive, individualized needs assessments for individual youth; use these 
needs assessments as the basis for service planning or case management; have 
developed formal, institutionalized interagency linkages; and conduct standard follow- 
ups with agencies to which referrals are made to ensure accountability. Not all 
programs ultimately visited met all five criteria. 

Programs Visited 

The nine programs in six locations that were ultimately selected represent a mix 
of program type, geographic location, and racial/ethnic groups served. They include 
the Belafonte-Tacolcy Center in Miami, Florida; Big Brothers/ Big Sisters of Greater 
Miami in Coral Gables, Florida; Chins Up Youth Care Homes in Colorado Springs, 
Colorado; Garfield Youth Services in Garfield County, Colorado; I Have a Future and 
Oasis Center in Nashville, Tennessee; Sunset Park-Center for Family Life in Brooklyn, 
New York; Teen Connections in the Bronx, New York; and Communities in Schools in 
Houston, Texas. These nine programs can be characterized as follows: 

• Age Range : Between 50 to 100 percent of clients served by these programs are 
age 10 to 15. 

• Gender. One program serves only girls; the remainder serve both boys and girls, 
but tend to have more boys. 

• Race /Ethnicity: Two programs serve almost entirely African-American youth, 
two serve mostly white youth, one serves mostly Hispanic youth, two serve a 
mixed group of Hispanic and African-American clients, and two have very 
ethnically mixed groups of users. 

• Focus of Activities /Services: Three programs focus their efforts mostly or 
exclusively on the youth themselves, but may assist a youth's family if it 
becomes apparent that help is needed; three programs focus on youth in some 
of their activities and place a heavy emphasis on involving the families of youth 
in other components of the program (e.g., for "caseload" clients); three 
programs have some activities mainly for youth, some services that involve 
youth and their families, some offerings for any interested community member, 
and an overarching goal of changing and empowering the whole community. 
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• Program Model The nine programs include one mentoring program, one 

program focusing on a geographically defined community, one operating almost 
entirely in the schools, three operating in both schools and the community, and 
three that are community-based. Five of the programs use case management 
and three offer crisis-oriented, short-term services. 



Program Descriptions (Chapter 5) 

The nine programs visited are each described using the following categories: brief 
history; current mission, goals, and objectives; service configuration; current 
clientele/users; type and makeup of SI network; funding sources; and evaluation. 



Cross-Program Issues (Chapter 6) 

Chapter 6 summarizes the findings from site visits with respect to the following 
issues: 



• Clarity about who is and who is not a client; 

• Client risk levels and their implications for program service offerings and for 
evaluation; 

• Program orientation toward strengthening families and/or neighborhoods; 

• The cultural context of program operation; 

• Scope and variety of service delivery, and the meaning of comprehensiveness as 
programs see it; 

• Service integration issues, including the scope and variety of networks and SI 
arrangements, history and evolution of SI, perceived impacts, and difficulties 
encountered and ways of handling them; 

• Program choice and tradeoffs with respect to client age range, 
prevention/treatment orientation, activities/services orientation, youth-family- 
community orientation; and 

• Evaluation issues, including program interest in and perceived pay-offs from 
evaluation, past history of evaluation activities, level of documentation 
currently available, and our perceptions of the feasibility of a multi-program 
evaluation with programs such as these nine we visited. 

Below we briefly summarize the site visit findings concerning service integration 
and evaluation issues. 
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Service Integration 



The site visits confirmed our initial view that programs use a variety of 
configurations to facilitate access to services and augment service delivery to program 
clients. Programs use both formal and informal arrangements. For instance, in the 
formal category, some programs: have staff from other agencies come to deliver a 
service either on a permanent or a scheduled basis, contract with other community 
agencies to provide services or join a multi-agency team that meets regularly to handle 
clients who need services from several agencies, and have contracts to provide 
services to clients of other programs on the site of the other program. In the informal 
category, some programs rely on consciously worked out relationships between 
program caseworkers and referral agencies to improve clients’ chances of getting 
needed services. These informal links are important because they are more common 
than formal SI arrangements. But because such networks often break down when 
key staff leave, they are no substitute in the long term for formal commitments. 

Finally, programs use different types of volunteers to expand their service options. 
Mentoring was the most common volunteer activity, though one program we visited 
provided emergency shelter through host homes with volunteer families and two 
arranged with volunteer members of the business community to provide services. 

• Information Sharing. We found that most of the programs with formal SI 
linkages (four of the nine programs visited participate in formal SI efforts) have 
worked out arrangements for release of information as needed, usually on a 
case-by-case basis. For instance, some programs release information 
temporarily for the purpose of having a multi-agency team design a client plan, 
but the releases are not general and do not go beyond the framework of that 
specific plan. 

On the other hand, programs that rely on informal cross-program service 
delivery mechanisms report that information sharing is a continuing problem. 
Even when they are trying to get help for a specific client, they say they cannot 
name the client so as not to violate the client’s privacy. Also, because of the 
informality of arrangements, information that needs to be shared is not shared 
because no feedback mechanism exists to assure that a referring person ever 
learns what happened with a referral. 

• Perceived SI Benefits. The programs with well-functioning SI arrangements cite 
several benefits, including: clients receive both an increased number of services 
and more appropriate services, participating agencies follow through on their 
commitments with greater speed and thoroughness, youth are much less likely 
to fall through the cracks, and staffing patterns stabilize because the programs' 
community-building philosophy is attractive to staff and increases their 
commitment to the program. 

• Difficulties Encountered. All of the programs we visited try in various ways to 
live with the disadvantages of categorical funding described above. The 
greatest difficulties cited other than those inherent in the current system of 
service funding are related to "turf' issues—between agencies, between program 
staff and staff of an agency with which they want to work, and between ethnic 
groups. Between-agency issues include several agencies competing for the 
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same dollars to develop similar programs; different agencies with control over 
some of the same youths not agreeing over the best approach, and therefore not 
willing to commit resources to the case(s); and different agencies having 
different goals for the program. 

Agency-to-program and program-to-agency tensions included disciplinary 
differences in approach engendering hostility or distrust, a key person in an 
agency being threatened by a program person’s expertise, and insensitivities 
with regard to peak workload periods. 

One program cites ethnic tensions in their larger service community over 
whether agencies affiliated with and serving particular ethnic groups would get 
their own resources or would have to be under the control of agencies affiliated 
with different ethnic groups. 

A final problem cited involves interactions within an agency on SI. Programs 
that have tried SI without commitment from both agency directors and line 
workers have run into difficulties. 

Conceptualizing SI More Broadly 

This project began with a view of service integration that is client-driven. It 
assumes that an agency has clients with service needs that it cannot meet entirely 
with its own resources, and that it becomes involved in formal interagency linkages to 
access services for its clients. We have learned that this view of SI is quite narrow 
and formal. It does not encompass several of the situations found during site visits, 
which appear to the researchers to epitomize an ideal of SI as service development 
and community coordination. Several programs we visited make themselves available 
to develop services as their need is manifested in the community. If the program itself 
or other agencies with youth-serving responsibilities identify major unmet needs, the 
community of agencies can negotiate exactly what is needed, who can best provide it, 
how the various agencies in town will relate to the new service, and other similar 
issues. These agencies serve as mortar for their community networks— they hold them 
together, fill in the gaps, and facilitate smooth service delivery whether through their 
own services or the services of other agencies. They may do relatively little through 
formal or even informal referrals of their clients to other agencies, yet they help create 
a truly integrated service delivery system. 

Evaluation Issues 

Two key conditions determine the readiness or "evaluability" of a program for an 
outcome evaluation; willingness and capability. Willingness refers to a program’s 
interest in evaluation and perception that evaluation can help advance program goals. 
Capability refers to a program’s current resources for evaluation— the skills of its staff, 
its data collection capabilities, and its commitment of staff time to evaluation 
activities. An important factor affecting both these conditions is the program’s history 
of participating in evaluation efforts, since this experience will contribute to staff 
attitudes about the experience and to existing capabilities. Among the nine programs 
visited, those with the highest levels of capability are usually those with more positive 
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attitudes toward evaluation. But in some programs the two dimensions do not exactly 
correspond. 

In terms of willingness, most of the programs show an interest in doing more 
evaluation research and, in particular, assessing program outcomes. Many directors 
specifically indicate that they want to do some form of longitudinal follow-up of their 
clients as an indicator of their program’s success. The enthusiasm of some programs 
relates to earlier positive experiences with evaluation studies. Among programs that 
appear more hesitant about evaluation research, one cites a bad experience it had 
with the evaluator for the national demonstration program of which it was a part, and 
one cites its concern about an evaluation’s ability to reflect the complexity of client 
experiences in the program. This program thinivo that the rather cut-and-dried 
approach to outcome assessment used by one evaluation did not do justice to either 
their services or the benefits their clients derived from the program. 

In terms of capability, the programs visited can be grouped into low, moderate, 
and high capability. Low-capability programs lack the existing resources required for 
an evaluation study, including staff knowledgeable about evaluation research, 
sophisticated information systems, and a central unit or department responsible for 
putting information together. These programs have some trouble tracking the 
involvement of outside agencies, which is an important component of documentation 
for SI types of programs. 

Moderate-capability programs maintain some form of a computerized database 
system into which service and client statistics are entered regularly. Some of these 
programs still rely on the executive director to analyze the service statistics, but 
generally top management is supported by volunteers and staff who complete the 
forms and do the initial tabulation of the information. Moderate-capability programs 
have the ability to use the documented information for the purposes of planning and 
internal evaluation. Such programs sometimes have quite specific plans for improving 
their evaluation capability. While these programs have strong interest in evaluation, 
some feel resources available for evaluations are insufficient. 

High-capability programs have highly sophisticated management information 
systems and staff specifically assigned to do the data entry, compilation, and 
summary statistics. They usually are conducting or have conducted some form of 
evaluation. They possess a high level of readiness, even though some have had some 
negative experiences with prior evaluation research. All of these programs clearly 
indicated that any costs incurred by doing evaluation research were more than 
compensated for by the benefits of the information obtained. 

The high-capability programs appear the most ready, and with some additional 
resources the moderate-capability programs may also be helped to participate in a 
multi-program evaluation. However, at least two issues need to be addressed in order 
to design an outcome evaluation that includes some or all of the sites visited and that 
will identify the effects of comprehensive service provision and SI models. 

The Jirst issue concerns the choice of comparison or control groups. It is 
not clear from the site visits how all programs may be able to identify a 
potential group of non-intervention clients. Although community-based 
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controls may be formed, there are the risks of contamination with the 
program participants and the possibility of these youth and families 
entering the programs themselves at some point. One potential avenue 
for the choice of control or comparison group is to draw these individuals 
from a matched sample living in an adjacent community that does not 
have such a program. 

• The second issue is how to resolve the variability in information currently 
documented by programs, particularly client risk level information and 
service provision characteristics that would be amenable to 
classification. It would also be important to identify a standard minimum 
data set that all sites provide for the cross-site analysis. 



All programs involved in a cross-site evaluation should become involved in this 
decision-making process so they develop ownership and positive attitudes toward a 
cross-site evaluation. Given the special features of these programs, the measures 
should not just assess indiviuual changes, but should also identify the effects of the 
programs on the community and on the interagency service delivery network. 



Summary and Implications (Chapter 7) 

The literature on at-risk youth and programs that serve them indicates that a 
comprehensive approach has the best chance of helping youth avoid negative 
behaviors and outcomes. SI is one way to increase the comprehensiveness of program 
offerings by facilitating access to services available in the community that a program 
does not itself provide. The programs we visited are all complex, all strive for a high 
degree of comprehensiveness, and most are involved to some degree in service 
integration to achieve it. 

To us, the most striking implication of this project’s findings is the need to 
conceptualize service integration more broadly. We think it is important to recognize 
the efforts that some programs make to develop their community’s capacity to serve 
youth, by identifying and working to develop services to address unmet needs. Also 
exciting was learning about Joint Initiatives in Colorado Springs— a service integration 
effort with the highest level of commitment from all relevant local agencies. 

Another important finding of this project is that these agencies have a veiy strong 
interest in conducting evaluation research, but most do not have the resources to go 
beyond the usual program statistics to assess outcomes or the role of particular 
services in achieving those outcomes. Almost all programs are interested in 
participating in outcome evaluations if they have adequate resources and technical 
assistance. They also feel it is important that any evaluation design reflect the 
complexity of their program activities and the many ways that youths, families, and 
community members may participate in them. 
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CHAPTER 1 



INTRODUCTION 



This report presents the results of a project to examine programs serving younger 
adolescents, aged 10 to 15, who may be at risk for participating in negative behaviors 
or experiencing negative outcomes. Some estimates suggest that as many as half of 
today’s youth run a moderate to high risk of experiencing school failures, or 
participating in early sexual activity, alcohol and drug use, and criminal behaviors. 
Circumstances of living in poor households, and especially in high-poverty 
neighborhoods, or living in abusive families, families affected by chemical dependency 
or other dysfunctions increase the risk for youth, as does associating with peers who 
engage in risky behaviors. 

The age range of interest in the project, 10 to 15, suggests that programs 
focusing on these youth may take an approach heavily geared toward primary 
prevention, but many programs will also offer traditional treatment and other 
intervention services. Traditional approaches to service delivery for youth, both 
prevention and treatment services, usually focused on only one type of problem 
behavior, from the point of view of one formal system. Thus, the schools worried 
about school failure and dropout; health agencies worried about sexuality, pregnancy, 
and childbirth; and criminal justice agencies worried about delinquency. As the 
results of many years of program impact evaluations became known, it has become 
increasingly clear that single-focus programs of this type may not be the most effective 
way to help youth. Increasing attention is being paid to programs capable of dealing 
with the whole child, and preferably also the child’s parents and neighborhood. This 
broadening of service perspective has implications for how programs work. 
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A major focus of this project has been to learn more about the service 
configurations developed by programs trying to provide more comprehensive services 
to youth in a coordinated way. Specifically, we have been interested in the ability of 
programs to deliver comprehensive services through service integration-formal 
cooperative networking with other agencies to assure full and efficient service delivery. 
We have also inquired into the relationship between comprehensiveness and service 
integration, and the barriers to providing comprehensive services through a service 
integration approach. 

The objectives of this project are to: 

• Document how comprehensive, integrated services are delivered to at-risk youth 
between the ages of 10 and 15; 

• Identify effective methods of providing comprehensive, integrated services for this 
population: 

• Identify barriers to providing comprehensive services, and means of facilitating 
service integration for at-risk youth; 

• Examine the role of Federal, state, and local government and the non-profit sector 
in impeding or facilitating service integration for at-risk youth; 

• Examine the extent to which simple lack of services, or insufficient service 
capacity, is implicated as a barrier, in comparison with eligibility, regulatory, 
jurisdictional, and other factors; and 

• Identify issues for further research on the provision of comprehensive, integrated 
services for at-risk youth. 



To pursue these objectives we reviewed literature on the meaning of risk, the 
prevalence of risk behaviors among youth, and the success of traditional programs 
serving youth. We also examined issues related to evaluating programs for at-risk 
youth. We then conducted site visits to nine programs in six locations, using the 
visits to explore the issues raised by the literature review and evaluation issues. 
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The remainder of this report is organized as follows. Chapter 2 presents a review 
of the literature on youth at risk. This is a much-abbreviated version of a larger 
report, "Youth at Risk: Definitions, Prevalence, and Approaches to Service Delivery" 
(Resnick et al. 1992), available as a separate publication. Chapter 3 examines issues 
that may arise if one wants to conduct formal evaluations of programs for at-risk 
youth, and also is available as a separate publication, "Evaluation Issues for Programs 
Serving Youth at Risk" (Burt and Resnick, 1992). Chapters 4-6 present the results of 
our site visits. Chapter 4 describes the purpose of the visits, how sites were selected, 
how site visits were conducted, and a brief overview of each program. Chapter 5 
presents program-by-program descriptions covering each program’s history; current 
mission, goals, and objectives; service configuration; current clients or users; type and 
makeup of the service integration network; funding sources; and evaluation activities 
and attitudes. Chapter 6 examines the most important cross-program issues 
identified in the site visits: identifying "clients"; client risk levels; orientation toward 
youth only versus also targeting families and/or neighborhoods; the impact of cultural 
context; the scope and variety of service delivery; service integration issues; choices 
and tradeoffs with respect to age of youth, prevention/ treatment or services/ activities 
orientation; and issues related to evaluation. Chapter 7 summarizes the project 
results and presents the implications of our findings. 
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CHAPTER 2 



YOUTH AT RISK: DEFINITIONS, PREVALENCE, 

AND APPROACHES TO SERVICE DELIVERY 

This chapter provides an overview of the extensive literature on at-risk youth, the 
services that exist to meet their needs and improve their life prospects, and efforts to 
create programs integrated across service systems. Our literature review relies heavily 
on secondary source material and interviews with recognized youth experts. 
Fortunately, several excellent reports have been completed recently that summarize 
the state of knowledge in the field. This chapter is not intended to provide an 
exhaustive literature review, but we think it presents a fair representation of the 
current collective wisdom about at-risk youth and service approaches. 

First, we examine current definitions of "adolescence" and "risk," in the latter 
case exploring their implications for identifying youth who might need services. 

Second, we summarize research on the prevalence of specific behaviors or outcomes 
that generally define the youth of interest, with particular focus on prevalence among 
10- to 15-year -olds where the data are available. Next, we look at traditional services 
for youth, which have tended to function through single-focus programs within single 
organizational systems such as education, corrections, or mental health. Finally, we 
examine the impetus for a more comprehensive and integrated approach to service 
delivery, and some of the issues involved in developing and providing such services for 
youth aged 10 to 15. 



ADOLESCENCE 

While the start of adolescence is most frequently identified as puberty, the end of 
adolescence is less clearly defined. Some experts and organizations are beginning to 
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increase the upper age limit to 24 years (World Health Organization 1986). Currently, 
American adolescents may cover the age range from 10 years to 19 years, although 
females typically mature earlier than males (Tanner 1972). Milestones in cognitive 
and emotional development as well as socioeconomic independence typically mark the 
end of adolescence (World Health Organization 1986). 

There is an increasing tendency to view adolescence as comprising two relatively 
distinct periods: "early adolescence" and "late adolescence." Early adolescence 
includes most pubertal change and roughly corresponds to the middle school or 
junior high school years (typically ages 10 to 15), while late adolescence includes the 
age range from 16 through 19 years (Santrock 1991). Although research results may 
not apply to adolescents of all ages, many studies do not provide separate breakdowns 
for the two age groupings (Hamburg and Takanishi 1989). When reports do make 
such a distinction, it is frequently not consistent; sometimes the cut-off age between 
early and late adolescence is 14, sometimes it is 15. 

Adolescence involves the task of forming a sense of identity accompanied by a 
cohesive set of personal values (Erickson 1968). During early adolescence, the young 
person forms a separate identity by negotiating relationships with parents and peers. 
This often happens at the same time that rapid physical changes are occurring. 
During the apex of the pubertal growth spurt, occurring among most early 
adolescents between the ages of 13 and 15 (Steinberg 1981), many adolescents 
experience increasing conflict between themselves and their parents. The appearance 
of such conflict during this period and its subsequent waning during late adolescence 
have caused many theorists to view adolescence as a time of "storm and stress" (Ross 
1972). In fact, it was previously believed that identity formation was facilitated by the 
child breaking the parent-child bond during this period of stress (Grotevant and 
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Cooper 1986). However, more recent evidence supports the view of adolescence as a 
gradual renegotiation of the parent-adolescent relationship (White, Speisman, and 
Costos 1983; Youniss and Smollar 1985). Adolescents are now viewed as 
transforming rather than abandoning their relationship with their parents while 
becoming more closely connected to a peer group (Youniss and Ketterlinus 1987). 
Adolescents generally need and want adult support when they are faced with 
important decisions, issues, or choices (W.T. Grant Foundation 1988). 

Widespread generalizations about the existence of a "generation gap" between 
"most" adolescents and adults have been fueled primarily by information about a 
limited number of individuals (Adelson 1979). Surveys have reported that there are 
actually few or no differences between the attitudes of adolescents and their parents 
on issues such as self-control, hard work, the law, long-term planning, and 
expectations for quality of life (Yankelovich 1974). An important theme in this chapter 
is that young adolescents do not comprise a homogeneous group, whose members are 
all at equally high risk for problem behaviors. As we shall see, levels of risk appear to 
be mediated by a set of environmental and individual antecedents that condition the 
nature of the relationship between risk status and negative outcomes. 

Although adolescence often involves some degree of experimentation, most 
adolescents experiment by trying out a variety of positive work and recreational 
identities before making a commitment to vocations, a career choice, or a given set of 
values (Marcia 1987). The development of a firm sense of identity during adolescence 
forms the groundwork for success as a fully-integrated member of society, which 
means being productive in work, meeting commitments to family and friends, and 
assuming the responsibilities of citizenship (Office of Technology Assessment 1991). 



Some adolescents may experiment with negative role identities involving such 
risky behaviors as gang membership, criminal and violent acts, early unprotected 
sexual intercourse, drug or alcohol abuse, and truancy from school. For those who do 
engage in risky behaviors, some still manage to become productive and successful 
adults, while others remain marginal members of society and become mired in welfare 
dependency, low levels of employability, drug addiction, and/or criminal and violent 
behavior. It is obviously important to be able to identify adolescents at varying levels 
of risk before problems become serious. 

THE MEANING OF RISK 

In this section we discuss the development of the risk concept and different 
definitions of the term. Then we integrate the findings of the empirical literature into 
a proposed model for defining different levels of risk among young adolescents. 

Three important trends in child development and prevention theory within the 
past fifteen years have contributed to the current interest in definitions of youth at 
risk. First, there has been acceptance and strong empirical support for "ecological 
theories" of human development since Bronfenbrenner published his comprehensive 
model for portraying the environment’s role in child and adolescent development 
(Bronfenbrenner 1979). New empirical evidence substantiates the influence of family 
processes, the peer group, social supports and community resources, neighborhood 
safety and quality of life, as well as the larger key social institutions affecting 
development such as the school, on the individual’s development (Kreppner and 
Lerner 1989). 

Second, findings from early intervention research conducted over the past ten 
years have also influenced current definitions of risk. Research from the Perry 
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Preschool Project (Berrueta-Clement et al. 1984) and the Yale Early Intervention 
Project (Seitz, Rosenbaum, and Apfel 1985) shows that early childhood interventions 
are able to reduce the negative effects of poverty and disadvantage on children’s 
school and social competencies, producing impacts still measurable after ten to 
twenty years. Broadly stated, these results suggest that the value of prevention 
extends well beyond the childhood years. 

Finally, the last five years have seen a shift toward viewing specific problems of 
adolescence-delinquency, substance abuse, pregnancy or parenthood, and school 
failure— as having common, rather than distinct, antecedent causes (Dryfoos 1990). 

These three factors— the ecological movement in child development, early 
intervention research, and the overlap between risk factors for problems of 
adolescence— have made people think more about assessing level of risk for future 
problems. 

Competing Definitions of "Risk" 

We now consider the various definitions of risk that have appeared in the 
literature. Risk implies probability, not certainty, that a youth will display problems. 
Implicit in defining risk is the attempt to predict the future course of events in a 
young person’s life. At the same time, a definition of risk must effectively identify 
those who are most likely to benefit from programs, services, or interventions. This is 
especially important when planning services during times of budgetary cutbacks, to 
make the most out of scarce resources. 

Exhibit 2.1 summarizes the various definitions of risk found in the literature and 
discusses their advantages and disadvantages for the delivery of services to youth 
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EXHIBIT 2.1: SUMMARY OF ALTERNATIVE DEFINITIONS OF RISK 
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at risk. The differences among definitions are often a matter of emphasis on 
particular aspects of risk, rather than being completely incompatible. 

The first row of Exhibit 2.1 represents risk definitions which rely on personal 
characteristics and aspects of an individual’s background to predict the likelihood of a 
future occurrence of negative behaviors and outcomes. These definitions focus on a 
single type of negative behavior—e.g., they tiy to predict substance abuse, or too-early- 
childbearing, or school dropout, or delinquent behavior, but not their co-occurrence 
and not "at least one of the above." This type of risk definition has long been popular, 
as has the tendency to focus on one negative behavior at a time. Most of the models 
developed from this type of risk definition do not have strong predictive power; they 
have not been able to identify a set of prior conditions that lead to specific outcomes 
with a level of precision sufficient to support programmatic decisions. Traditional, 
single-issue programs have frequently used this definition as a rationale for their 
program focus and the lack of precision in the definition affects the efficiency of these 
service delivery efforts to target those youth at varying levels of risk. 

The second row of Exhibit 2.1 represents definitions that assess "risk" on the 
basis of problem behaviors in which youth already engage. As a definition of risk, this 
approach is weak because we know with certainty that the behavior has happened. 
Further, by the time youth are identified by this type of definition as "high risk," they 
are beyond the point of needing simple prevention interventions. Programs will have 
to offer more intensive treatment, often with less hope of averting continuation of the 
behaviors and their consequences in the future. 

A variant and extension of the "risk is defined by behaviors" approach is one that 
attempts to estimate the Joint probability that youth will engage in at least one 
negative behavior or experience at least one negative outcome. Dryfoos (1990) is the 
most recent practitioner and synthesizer of this approach. She argues that because 
problem behaviors share common antecedent characteristics, all of these problem 
behaviors of youth are probably interrelated. Therefore different levels of risk can be 
defined according to the number and seriousness of multiple problem behaviors that a 



youth exhibits (e.g., school failure, substance abuse, delinquency, or pregnancy). She 
estimates that 25 percent of the adolescent population aged 10 to 17 may be 
considered to be at "high" risk for developing one or more of these problem behaviors. 
Another 25 percent are estimated to be at moderate risk and the remaining 50 percent 
of adolescents are considered to be at "low" risk. Unfortunately, these estimates of 
risk are flawed due to the methodological problems of the research used to create 
them. Generally, research does not specifically test the hypothesized overlap or co- 
occurrence of behaviors; since the research studies used as support were not designed 
to do so, results may be misinterpreted (Takanishi 1992). 

The final row of Exhibit 2.1 represents definitions that emphasize the 

environment that surrounds the youth, rather than the youth’s behavior per se. For 

these definitions, youth are at risk because they live in "risky situations or 

environments," not because they engage in "risky behavior" (Takanishi 1992). Living 

in dangerous neighborhoods, in inadequate housing, with negative role models from 

peers and adults, without sufficient parental support and monitoring, and with few 

opportunities for future employment, predisposes an adolescent to engage in those 

behaviors that place him/her at risk of developing serious negative consequences 

(Schorr and Schorr 1988; Primm-Brown 1992; National Network of Runaway and 

Youth Services 1991). This definition offers a compelling counterpoint to definitions of 

risk based on individual behavior, and suggests intervention strategies that target 

whole neighborhoods with massive prevention efforts. Interventions based on an 

environmental strategy will certainly reach many more neighborhood children than 

those who actually participate in negative activities. But that is the balancing act that 

programs face in deciding on their mix of prevention and treatment strategies. A final 

difficulty with the "risky environment" approach to defining risk is its potential for 

labeling all children in a neighborhood with a single stereotype. Officials may expect 

children from certain neighborhoods to misbehave or to fail, and may adjust their 

behavior and expectations accordingly, thereby creating the outcome they were trying 

to avoid. Adolescents may accept the label and participate more fully in the peer 
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culture surrounding the display of abnormal behavior (Goffman 1961). Finally, the 
ecological viewpoint downplays the fact that many risk factors and problem behaviors 
can be found among people of all income levels and communities and overlooks the 
fact that some youth from even the worst neighborhoods manage to avoid problem 
behaviors. Research documents the existence of factors promoting resilience in 
children exposed to substantial environmental risk, including: having personal 
characteristics such as higher intelligence, personal charm or optimism, being first- 
born, coming from smaller families with better birth spacing, having a supportive 
relationship with a caring adult (not necessarily a parent), and having access to social 
support outside the immediate family (Garmezy, Masten and Tellegen 1984; Mulvey, 
Arthur, and Reppucci 1990; Rutter 1979; Werner 1986, 1988; West 1977; West and 
Farrington; 1973). 

The different approaches to defining at-risk youth presented above are not 
incompatible. Youth who engage in multiple problem behaviors are more likely to 
come from environments that place them at greater risk. An emergent perspective 
focuses on "health" defined broadly to encompass mental and social as well as 
physical aspects (Office of Technology Assessment 1991). According to this view, 
environments or behaviors are "high risk" because they have serious health 
consequences, which include anything preventing the individual from becoming a fully 
functioning member of society. Factors in the youth’s family, school, community, and 
larger societal environment that influence his or her physical, mental and social 
health lead to greater or lesser degrees of risk for developing problems (Office of 
Technology Assessment 1991). This more complete and integrated perspective for 
assessing risk reflects the nature of the paradigm shift away from single-problem 
views of adolescence and serves as an organizing principle for our proposed model of 
defining risk in adolescence. 
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A Conceptual Framework for Defining Risk 

The definition of risk requires a model that integrates the assumptions about 
cause and effect and the nature of the associations between environment, individual 
behavior, and health outcomes. We propose a conceptual framework that synthesizes 
the diverse literature on adolescent development, problems of adolescence, and 
theories of prevention. This framework takes into account the common antecedents of 
many adolescent problems. It allows for an assessment of risk geared specifically to 
young adolescents, which emphasizes the early signs of dysfunction rather than the 
onset of negative or destructive consequences. 

The risk definition that we propose consists of four components—risk 
antecedents, risk markers, problem behaviors, and outcomes--and can be stated as 
follows: 

The presence of negative antecedent conditions (risky environments) 
which create vulnerabilities, combined with the presence of specific 
negative behaviors, define a youth’s level of risk for incurring more 
serious consequences (risk outcomes). Early indicators of risk may be 
found in risk markers— indicators available from public records that 
signal risk. 

Exhibit 2.2 presents a schematic representation of the risk model, whose four 



components are: 

• Risk antecedents: Those environmental forces that have a negative impact on 
the developing individual by producing an increased vulnerability to future 
problems in the family, school, or community. Based on our review of the 
literature, there appear to be three critical risk antecedents for early adolescents; 
poverty, neighborhood environment, and family environment. 

• Risk markers: These are visible indicators of behavior, available from public 
records. Previous research suggests a consistent relationship between these 
behaviors and risk antecedents, and a well-defined link with increased 
vulnerability and the onset of potentially negative behavior. We have selected 
two indicators that are consistently identified as markers for all problem 
behaviors of adolescence: poor school performance and involvement with child 
protective services, including out-of-home placement in the foster care system. 
These two have particular policy relevance because they can be observed in the 
records of public systems, and allow program planners to target the youth at 
greatest risk. 

• Problem behaviors: These are defined as activities that have the potential to 
hurt youth, the community, or both. Research has identified these behaviors as 
those most likely to occur in youth who earlier displayed risk markers, or who 
were living under risk antecedent conditions. We have chosen those behaviors 
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that have most consistently been identified in the literature as signalling 
potentially more serious consequences for youth in the future, including, 
engaging in: early initiation and practice of sexual behavior: truancy or absence 
from school: running away from home (or from an out-of-home placement): early 
use of tobacco, alcohol, and other drugs: and associating with delinquent peers. 

• Risk outcomes: These are clearly injurious conditions that have negative 
consequences for a youth’s future development as a responsible, self-sufficient 
adult. The risk outcomes of primary concern include teenage pregnancy/ 
parenthood, homelessness, involvement in prostitution, alcoholism and/or drug 
abuse, delinquency and criminal behavior, school dropout, AIDS, chlamydia and 
other sexually-transmitted diseases, physical and sexual abuse, and various 
morbidity and mortality conditions (hepatitis, tuberculosis, pneumonia, 
accidents, suicide, homicide). 



At minimum, we would consider a young adolescent to be at "high risk" if he/ she 
grew up under any of the antecedent risk conditions and is currently displaying one 
or more of the risk markers. "Moderate Risk" would be assigned to those youth who 
are either living under any of the antecedent conditions or are currently displaying 
one or more of the risk markers. "Low risk" would be assigned to those young 
adolescents who are not living in negative antecedent conditions and who are not 
displaying those negative behaviors which are risk markers. This definition of risk is 
specifically geared towards the younger age group of adolescents, from 10 to 15 years 
of age, because it relies on early markers of risk, which are more likely to be evident 
among this age group than serious negative outcomes, and which should be the focus 
of prevention efforts. Of course, treatment efforts should be addressed to any 10- 15- 
year-olds who already exhibit serious risk behaviors or experience negative outcomes. 

The model is meant to reflect the prevailing view in the literature to date 
suggesting a confluence of factors, including increased vulnerability, multiple 
causation, and the transaction between the environment and the individual (Sameroff 
and Fiese 1989). The model is not meant to imply any strict causal connections. 
Certainly the literature indicates that a youth is more likely to display risk markers if 
they also have risk antecedents, but markers may appear in youth with no 
antecedents, and youth with antecedents may display no markers. The same 
relationships pertain between antecedents and markers and the behaviors and 
outcomes displayed in the last two segments of Exhibit 2.2. Nevertheless, research 



does suggest that the antecedents do successfully predict the presence of markers in 
many cases, and that both antecedents and markers often predict negative behaviors 
and outcomes. 

PREVALENCE OP RISK ANTECEDENTS, MARKERS, BEHAVIORS 
AND OUTCOMES AMONG 10- TO 15-TEAR-OLD ADOLESCENTS 

In this section we summarize the available survey-based and/or population- 
based data which indicates the prevalence of risk antecedents, risk markers and 
problem behaviors/risk outcomes in the 10- to 15-year -old population. The variables 
chosen are those most consistently related to risk for young adolescents in the 
literature. Estimates of the prevalence of at-risk youth in the population of 10 to 15- 
year -olds using the above definition of risk (all four elements) would ideally be based 
on data revealing how many youth experienced each problem behavior or risk 
outcome. No single source has evaluated the prevalence of the entire range of 
possible problem behaviors among adolescents, the covariation among problems, or 
the likelihood of outcomes arising from specific behaviors (Office of Technology 
Assessment, 1991). In fact, although we have dealt separately with problem behaviors 
and risk outcomes, as requested by ASPE, the elements in these two categories are 
frequently confused or confounded in the literature. The most methodologically sound 
prevalence estimates come from studies of individual problem behaviors and health 
problems. However, few studies properly disaggregate the young adolescent (10 to 15 
years old) from the older adolescent (16 to 19 years old) sub-groups. The following 
discussion gives the prevalence of various problems among youth, with particular 
emphasis on those aged 10 to 15 if available, and for 10 to 14 year-olds in most 
instances. 

Prevalence of Risk Antecedents for Young Adolescents 

There is general agreement that at least one of two underlying living conditions 
are common to most adolescent problem behaviors: poverty and family dysfunction. 
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Further, when neighborhoods are characterized by very high poverty rates (underclass 
neighborhoods), the neighborhood itself contributes to the risk that youth will 
experience harmful outcomes. These factors are considered antecedents because they 
exist prior to problem behaviors or negative outcomes in any given youth, and there is 
empirical support for their value in predicting youth problems. Many researchers 
have identified clusters of adolescent high-risk behaviors that appear to stem from a 
complex interplay of multiple antecedent factors (Botvin 1985). Thus, the same 
outcomes may arise from different combinations of risk factors; one cannot predict 
risk without considering both the individual and the environment with which the 
individual interacts. 

Poverty 

According to data from the March 1988 Current Population Survey compiled by 
the Office of Technology Assessment (1991, Vol.l, 1 13-116), 26.7 percent of all 
American youth (or 8.27 million) aged 10 through 18 in 1988 lived in poor or near- 
poor families. These same data show that certain groups of racial and ethnic minority 
youth are more likely than white, non-Hispanic youth to be living in poor or near-poor 
families. In 1988, 17.3 percent of white youth lived in poor or near-poor families, 
compared with 52. 1 percent of African-American youth, 49 percent of Hispanic youth, 
32 percent of Asian youth, and 51 percent of American Indian and Alaskan Native 
youth. In addition, some parts of the countiy have a higher percentage of youth living 
in poor or near-poor families compared with other parts of the country. The South 
has a higher percentage of youth living in poor or near-poor families compared with 
the West or North. Despite the stereotype of poverty being a predominantly inner-city 
problem, a substantial percentage of poor families with children live in rural (30 
percent in 1987) or suburban (28 percent in 1987) areas (Bane and Ellwood 1989). 

Youth living in female-headed families are at much greater risk of being poor or 
near-poor than youth living with both parents or those living with their father only 
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(Bane and Ellwood 1989), and female youth who bear children out of wedlock run the 
greatest risk of living in poverty for many years. 

Furthermore, there are a variety of health and behavioral consequences for youth 
living in poor or near -poor families that increase their risk for problems. For instance, 
youth living in poor families are more likely to miss days from school due to illness or 
injury, thereby affecting their school performance (U.S. Department of Health and 
Human Services, Centers for Disease Control 1990). Finally, living in poverty is 
associated with an increased likelihood of early sexual activity and teenage pregnancy 
(Moore, Simms, and Betsey 1986). Youth living in poverty who become pregnant are 
less likely to have an abortion or to give their child up for adoption, compared with 
youth from less disadvantaged backgrounds (National Academy of Sciences 1989). 

Neighborhood 

Some research documents the effect of neighborhood on youth outcomes, in 
addition to the influence of family poverty or dysfunction. (Gibbs et al. 1988; U.S. 
Department of Health and Human Services, Centers for Disease Control 1990). Much 
recent research about "the underclass" is premised on the assumption that the 
concentration of poverty in central cities has created a situation that is a cultural and 
behavioral phenomenon as well as an economic one (Jargowsky and Bane 1990; 
Ricketts and Sawhill 1988; Wilson 1987). "Underclass" areas are characterized by 
high levels of many social problems including family dysfunction, high unemployment, 
and high welfare receipt. Some of the social problems associated with these areas are 
those affecting youth— high rates of school dropout, teenage unemployment, and 
teenage pregnancy and out-of-wedlock childbearing. At a minimum, more youth in 
these areas are exposed to the opportunity to participate in problem behaviors 
without having to look very far to find them. 
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Family Dysfunction and Lack of Parent Support/ Involvement 

Empirical research from an ecological model of development has consistently 
shown the importance of parental support and involvement as a critical mediator of 
child and adolescent development. Parental behavior can have negative effects, such 
as when parents are chemically dependent, neglectful, or abusive. Family dysfunction 
has been linked empirically to adolescent problem behaviors in many studies 
(Patterson, cited in Kumpfer 1989; Sroufe and Rutter 1984). Some parenting "styles" 
appear more likely to occur in dysfunctional families. "Authoritarian" parents are 
hostile, rejecting, strict and punitive, whereas "laissez-faire" parents are over- 
indulgent, permissive or neglecting. Both patterns are associated with adolescents 
who are less competent socially, have lower levels of self-esteem, and are more likely 
to display negative behaviors (Baumrind 1991). 

Typically, the "symptoms" of family dysfunction are often what brings a 
particular adolescent or family to the attention of social and community service 
agencies, including the juvenile authorities, courts, treatment agencies, shelters, and 
child protective services. One method of estimating the prevalence of these 
"symptoms" is through data available on several indicators of dysfunction: parental 
substance abuse, family violence, and adolescent maltreatment. 

Alcoholism and abuse of illicit drugs by an adolescent’s parents or siblings have 
been shown to significantly increase an adolescent’s vulnerability to becoming an 
alcohol or drug abuser (Springer et al. 1992; Thorne and DeBlassie 1985). Parents 
who abuse alcohol or other drugs spend less time positively reinforcing their children 
for good behaviors (Kumpfer 1989), and there is a greater risk for family violence in 
families with alcoholic parents, due to the parents’ failure to deal effectively with child 
discipline, which "sets into motion coercive interaction sequences that are the basis 
for training in aggression" (Patterson, cited in Kumpfer 1989). In 1988, there were 28 
million children of alcoholics, 25 percent of whom, or approximately 7 million, were 
under the age of 18 (Office for Substance Abuse Prevention, 1989). 
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As for family violence, the results of one study (Straus and Gelles 1986) indicate 
that all forms of parental violence against children aged 3 to 17 years remained 
relatively stable from l'975 to 1985 at 6.2 per 1 ,000, with a prevalence rate for child 
physical abuse of 2 to 4 percent of the population ages 17 years or under. Another 
study, analyzing child maltreatment cases known to community agencies by various 
age groupings, found that between 1979 and 1986, the number of cases per 1,000 
children both between the ages of 12 and 14 and between the ages of 15 and 17 
nearly doubled (U.S. DHHS, National Center on Child Abuse and Neglect. 1980, 
1988). 



Prevalence of Risk Markers Among Young Adolescents 

"Risk markers" for young adolescents are early signs that they engage in problem 
behaviors or experience negative outcomes. These markers generally tend to arise 
from the antecedent conditions already identified: economic disadvantage, poverty, 
and/or family dysfunction. There is general agreement that a young adolescent who 
displays poor school performance or is retained in grade is more likely to exhibit later 
problem behaviors. In fact, Dryfoos (1990) argues that poor school performance is the 
single most important marker for identifying those likely to be at high risk. A second 
marker in early adolescence for high-risk status is whether the adolescent is involved 
with child protective services or out-of-home placement as a result of abuse or 
neglect. 




Grade Retention and Poor School Performance 

For young adolescents, being retained in grade is the single most important 
predictor of school dropout, after controlling for ability (Feldman, Stiffman, and Jung 
1987). Those who are two or more years behind their modal grade are considered at 
the highest risk of dropping out. Census data for 1 986 reveal that for adolescents 
aged 10 to 15 years, 28 percent of whites, 57 percent of African-Americans, and 63 
percent of Hispanics are two or more years behind their grade level (U.S. Bureau of 
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the Census 1988). Not only are many 10- to 15-year-olds at risk for dropping out, but 
males are more likely to be retained in grade than females and, for most age and sex 
groups, the probability of being two or more grades behind is at least twice as high 
among minority children as among white children (U.S. Bureau of the Census 1988). 
Dryfoos (1990) estimates that 4.5 million 10- to 14-year-olds are behind grade, most 
by one year, but she estimates that .7 million of these adolescents are behind by two 
or more years, and, thus, at highest risk for dropping out. 

Although grade retention is one operational definition of poor school 
performance, it is also important to consider low school achievement. According to 
the 1990 National Assessment of Educational Progress, students in general were 
better readers in the 1980s than they were in the 1970s, but the mean reading profile 
of African-American and Hispanic 1 7-year-olds was only slightly better than the 
reading profile of white 13-year-olds. Nevertheless, having a high school diploma, 
even with a poor achievement record in school, significantly improves labor market 
participation (Young 1983), so the bottom line when it comes to poor school 
performance may be whether the outcomes result in dropping out of school. 

Family Breakdown 

When family dysfunction reaches the point of child maltreatment or neglect or 
when the adolescent is considered uncontrollable or engages in criminal behavior, the 
child welfare (or the criminal justice) system usually intervenes. The child welfare 
agency arranges placement for the adolescent in an alternative family or group home 
environment. This placement can be temporary while efforts are made to reunite the 
adolescent with the parents or more permanent when reunification of the family is not 
possible. Foster care is usually the placement of choice. Two-thirds of all children 
under 16 years of age who are in out-of-home placement are placed in families and 
the rest are sent to institutions, often because no suitable family home can be found. 
In 1985, 270,000 children were in foster care, of which 45 percent were between the 



ages of 13 and 18 years; disproportionate numbers were non-white and Hispanic 
(William T. Grant Foundation 1988). 

Foster care or alternative custody placement of an adolescent can be a precursor 
or marker for more serious consequences such as homelessness, delinquency, or 
substance abuse. A 1990 study reported that the more foster care placements an 
adolescent had experienced, the more difficulties he or she encountered in later life 
(Family Impact Seminar 1990). 

Prevalence of Problem Behaviors and 
Risk Outcomes in Young Adolescents 

Below we review the prevalence of particular problem behaviors among young 
adolescents. 

Early Sexual Behavior, Pregnancy, 

Parenthood, and Sexually Transmitted Disease 

As Diyfoos (1990) points out, once an adolescent engages in sexual intercourse, 
he/ she could be considered "at risk" of unintended pregnancies or births, especially 
when contraception is not consistently used. In 1988, one in three adolescent males 
(ages 15-19) and one in ten adolescent females reported having had intercourse before 
the age of 15. The rates were approximately double among African-American teens. 
Although the scope of the problem for the population of 10-15 year olds is 
considerably smaller than for older adolescents, the consequences are probably more 
serious. These younger adolescents are even less equipped to make pregnancy 
resolution and parenting decisions than their older counterparts. Moreover, 
pregnancies during early adolescence may signal sexual abuse (Moore, Nord, and 
Peterson, 1989). 

For adolescents of all ages, close to one in four (23 percent) of sexually active 
teens experience a pregnancy during any 12-month period (Dryfoos, 1990). As for 
births (as opposed to pregnancies), there is a large disparity in rates for African- 
American and white adolescents ages 10 to 14: for whites the rate is .7 births per 
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1,000 in 1989 compared to 5.0 births per 1,000 for African-Americans. For 15-17 
year-olds, the birth rate for African-American adolescents is nearly three times the 
rate for whites. 

There are a number of antecedent variables that predict increased likelihood of 
early sexual activity (Dryfoos 1990). Males who are African-American, living in low- 
income families, with parents who are not supportive and do not monitor their child’s 
activities, are more likely to initiate sex at an early age. In addition, children who Eire 
not involved in school activities, who have low expectations for school achievement, 
and who are influenced by friends in similar situations Eire also more prone to engage 
in early sexual activity. Finally, young adolescents who are typically low school 
achievers, belong to a peer group that accepts parenthood, and Eire from poor, female- 
headed families in which parents do not monitor their activities Eire more likely to 
become teen parents. 

Dryfoos (1990) estimates that 1.9 million adolescents between 10 to 14 years of 
age are at risk due to their early sexual activity. Approximately 300,000 adolescent 
females aged 10 to 14 years of age are likely to become pregnant; of these, one-third 
will become parents. 

Sexually Transmitted Diseases 

Even excellent contraceptive practice, if not supplemented with condoms, does 
not help prevent sexual transmission of disease. In the late 1980s, there was a 63 
percent jump over a two-year period in the rates of gonorrhea among young 
adolescents. The syphilis rates for this age group are equally alarming: for 10- to 14- 
year-olds, the 1987 syphilis prevalence rate represents a 75-percent increase from 
1977 (Office of Technology Assessment 1991). 

One of the more serious consequences for adolescents who develop an STD 
(particularly those with syphilis) is the increased likelihood of their becoming HIV- 
infected (Office of Technology Assessment 1991). In 1990, AIDS was the sixth leading 
cause of death among 15- to 24-year-olds, although cases of AIDS among adolescents 



aged 13 to 19 represented under 1 percent of all AIDS cases. The prevalence of HIV 
infection may give a more accurate indication of the potential AIDS problem within the 
youth population than does the count of reported AIDS cases, due to the long 
incubation period for AIDS. Data from Job Corps entrants, who are economically 
disadvantaged 16- to 21 -year-olds, show a seroprevalence rate of 3.6 per 1,000, ten 
times higher than among military applicants the same age. "remarkably high ... for a 
population so young and not specifically selected because of behavioral risk factors" 

(St. Louis et al., 1991). The high rate of HIV infection among younger females 
suggests that heterosexual transmission of HIV may be responsible rather than 
intravenous drug use, which is higher in males. 

Truancy and School Dropout 

Little adequate prevalence data exist to indicate the numbers of truant youth, 
either in total or by age. Furthermore, younger adolescents may not be adequately 
represented in truancy and dropout statistics if they are runaways, homeless, or if 
they have been suspended from school. Most of the antecedents of poor school 
performance discussed earlier in this paper are also relevant in predicting truancy 
and dropping out of school. 

Ten- to fifteen- year-olds may be at risk for school dropout, but the prevalence of 
risk in this population is not fully reflected in the dropout rate because school 
attendance is compulsory until age 16. Younger adolescents may virtually drop out of 
school through repeated truancy, suspension, or expulsion, but schools will still carry 
them as officially enrolled until their 16th birthday. 

Diyfoos (1990) argues that many expected outcomes of school failure may also 
function as antecedents or markers. For example, delinquent behavior, including 
truancy and minor offenses during early adolescence, typically occurs prior to actual 
school dropout or failure. But once youth leave school they are more likely than those 
who remain to commit serious offenses. 
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Homelessness and Running Away 

Data on the actual numbers of homeless adolescents ages 10 to 15 who are living 
with their families are not available. One report estimates that 12 percent of homeless 
families include an adolescent between 13 and 16 years of age and another 36 percent 
of homeless families have a child between the ages of 6 and 12 years (U.S. Congress, 
General Accounting Office, 1989). Another study found that 26.6 percent of families 
living in homeless shelters had children between the ages of 1 1 and 1 7 years (Miller 
and Lin 1988). 

Data on adolescents who are homeless and living on their own (unaccompanied 
minors) must be estimated separately from data on youth living with their homeless 
families. The National Network of Runaway and Youth Services (1991) differentiates 
among "runaways," who are away from home at least overnight without parental or 
guardian permission, "homeless youth," who have no parental, substitute foster, or 
institutional home, and "street kids"--long-term runaways or homeless youths who 
have been able to live "on the streets," usually through illegal activities. Among the 
homeless youth are "throwaways" or "pushouts" who have been told to leave the 
parental household or who have been abandoned or deserted by their parent or 
guardian. Little is known about unaccompanied homeless youth on a national basis, 
since no studies to date have solved the methodological problems involved in 
obtaining such data. Homeless youth in special surveys, (summarized by Rotheram- 
Borus, Koopman, and Ehrhardt 1991), are disproportionately African-American or 
Hispanic, from lower socioeconomic backgrounds, and from single-parent families. 

On the street, they are quite likely to be victims of robbery and of physical assault, 
including rape. Approximately half are not enrolled in school and about half of those 
in school have learning or conduct problems. Observed rates of depression for 
unaccompanied homeless youth range in different studies from 26 percent to 84 
percent and are significantly higher than clinical samples of adolescents who are not 



runaways. 



Homeless youth who live on their own are more likely to engage in sexual risk 
behaviors than are non-homeless adolescents, dramatically increasing the risk of HIV 
infection among this group (Rotheram-Borus, Koopman, and Ehrhardt 1991). 

Between 50 percent and 71 percent of street youths have a sexually transmitted 
disease; pregnancy and motherhood are significantly higher among homeless girls; 
and the average age at first intercourse is about 12.5 for homeless youth, about two 
years earlier than for other adolescents. Homeless youths are also five times more 
likely to meet the criteria for a diagnosis of drug abuse than are non-homeless 
adolescents. 

There is a good deal of national data on use of tobacco, alcohol, and other drugs 
among adolescents, thanks to a number of different surveys, including the 1987 
National Adolescent Student Health Survey, 1985-1991 NIDA National Household 
Surveys on Drug Abuse, the 1980-1991 High School Senior Surveys (HSSS), which in 
1991 also surveyed 8th and 10th graders, among others. In general, the data reveal a 
number of interesting patterns. Contrary to popular belief, African-American teens 
were less likely than adolescents from any other racial or ethnic groups to report the 
use of an illicit drug, regardless of whether the measure was lifetime, annual, or past 
month (the same Is true for alcohol use). Hispanic adolescents, particularly females, 
were more likely to use illicit substances, particularly alcohol. Further, the data from 
several national studies converge to indicate that teenage use of all drugs and some 
specific drugs (e.g., cocaine) has been declining since the early 1980s. As noted by all 
studies, use of these substances does not necessarily mean abuse. Furthermore, with 
the decline in the acceptability of substance use and actual decreases in the 
prevalence of substance use among adolescents, the remaining users may represent a 
population who are either already addicted or addiction-prone (U.S. Department of 
Health and Human Services, 1992). An important fraction of the youth population 
already abuse alcohol and drugs. For heavy alcohol abuse, this may be as high as 25 
percent. A youth's age at first use of alcohol is often used as a marker for later 
alcohol abuse as well as for later use of other drugs (Welte and Barnes, 1985). 
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According to the 1990 Youth Risk Behavior Survey, 33.6 percent of all students 
sampled from grades 9 through 12 had first consumed alcohol before age 12 (Centers 
for Disease Control, 1991). 

Associating with Delinquent Peers, 

Delinquent and Criminal Behavior 

In general, "delinquent" acts are either criminal offenses or status offenses. 
Criminal offenses are those acts committed by minors that would be considered 
violations of criminal law if committed by an adult, such as murder, rape, assault, 
robbery, theft, burglary, or vandalism. Status offenses are acts committed by minors 
that would not be offenses if committed by an adult, for example, running away from 
home, truancy, alcohol use (Office of Technology Assessment 1991). 

Estimates of "delinquent" behavior and "delinquent" youth come from a variety of 
sources, including rates of offenses and arrests provided through the Uniform Crime 
Reports, self-reported delinquency and criminal behavior from the National Youth 
Survey, and victimization rates from the National Crime Survey. Several data sources 
are required to pinpoint delinquency because no single source provides an adequate 
measure of delinquency among adolescents (Elliott, Dunford, and Huizinga 1987; 
Huizinga and Elliott 1986). 

Older data from the 1976 to 1980 National Youth Survey indicate that a large 
majority of U.S. adolescents commit minor offenses at least once and that a small 
minority of adolescents also commit serious offenses at least once (Elliott et al. 1983). 
In the National Youth Survey, 21 percent of youth in the sample reported having 
committed at least one serious offense in 1976 (Elliott et al., 1983). The minority of 
adolescent offenders who commit many serious offenses are the adolescents most 
likely to continue criminal behavior as adults. Compared to nonchronic offenders, 
chronic Juvenile offenders were more likely to have begun delinquent behaviors at an 
earlier age, to have continued to commit them, and to commit a variety of offenses 
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rather than specializing in a single type of offense (Blumstein et al. 1986; Farrington 
1983; Farrington and West 1989). 

A host of factors are associated with the greater risk of delinquency. With 
respect to 10- to 15-year-olds, it is important to focus on those risk factors that occur 
earlier and are most likely to predispose youth to later delinquency, rather than 
concentrating on youth who already have criminal records, since fe\fter in this age 
group have actually committed serious offenses. Antecedent factors associated with 
the predisposition or risk of delinquent behavior include demographic characteristics, 
neighborhood and community (e.g., extent of anti-social peer culture therein), and 
family and individual characteristics (e.g., learning disabilities, associating with 
delinquent peers, drug or alcohol abuse). However, it should be noted that a small 
number of adolescents become delinquent without any identifiable risk factors in their 
background, which testifies to the lack of adequate understanding of delinquency 
(Rutter and Giller, 1984). 

Adolescent Mortality and Causes of Death 

Many of the antecedents and problem behaviors we have discussed increase the 
probability that a young person will die before reaching the age of 20. For young 
adolescents, the leading cause of death is injuries, including injuries from accidents 
and from suicide and homicide attempts. Suicide and homicide accounted for 79 
percent of all injury- related deaths in 1987 (Office of Technology Assessment 1991). 
Injury death rates for youth aged 10 to 14 decreased from 23.6 to 16.3 deaths per 
100,000 between 1950 and 1987, while rates for older adolescents ages 15 to 19 
increased over the same period (Office of Technology Assessment 1991). 

A host of factors predict accidental death from injuries, including demographic 
characteristics (age, gender, race and ethnicity, and social class); risk-taking behavior 
(alcohol or drug abuse, failure to use safety belts, and failure to use bicycle or 
motorcycle helmets); and stressful life events (suspension from school, failing a grade 
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level, difficulty getting a summer job, breaking up with a boyfriend or girlfriend, and 
the death of a grandparent). 

Dryfoos (1990) reported suicide and homicide rates for youth 12 to 17 using data 
from the National Center for Health Statistics. From 1980 to 1986, rates increased in 
each of four groups (African-American and white adolescents in age groups 12-14 and 
15-17), with the largest increases reported among African-American 12- to 14-year- 
olds. Overall, 7 percent of deaths in the 12- to 14-year-old group were due to suicide 
in 1986 and 6 percent were due to homicide. However, African-American male teens 
are 5 to 6 times more likely to die from homicide than white male teens, and African- 
American female teens have 2 to 3 times the death from homicide rates of white 
female teens. The suicide rate among American Indian adolescents was four times 
higher than the rate for all other races among the 10- to 14-year-old population 
reported for the same year (Office of Technology Assessment 1991). 



Summary 

This literature review has been organized following the conceptual framework of 
risk described earlier, containing four components: risk antecedents, risk markers, 
problem behaviors, and risk outcomes. The framework is based on evidence showing 
that many problem behaviors share similar antecedents. Dryfoos (1990) points to six 
common characteristics that predict high risk of the four main problem behaviors of 
adolescence— substance abuse, delinquency, school dropout, and pregnancy or 
parenthood. The adolescent at greatest risk is one who: 1) initiates the behavior early; 
2) has low expectations for education and school grades; 3) is antisocial, acting out, or 
truant; 4) has low resistance to peer influences and associates with friends who 
participate in the same risky behaviors; 5) has poor support and monitoring from 
parents and is unable to communicate with parents; and, 6) lives in an urban poverty 
area. 

Despite the apparent overlap in antecedents and markers, it is difficult to develop 
a composite estimate of the degree to which adolescents run a high, moderate, or low 
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risk for engaging in problem behaviors or experiencing risk outcomes. At the 
population level, perhaps the simplest approach is to base a rough estimate on readily 
available and reliable national data such as the poverty rate, which puts the 
proportion of youth at risk at about 21 percent, since 21 percent of children live in 
poor households. Minority status is associated with higher risk because it is 
associated with poverty, especially poverty in neighborhoo ' - with very high poverty 
concentrations (21 percent of poor children for both African-Americans and Hispanics, 
compared with 2 percent of white children-Jargowsky and Bane 1990). 

The simple population estimate based on poverty or neighborhood is very rough, 
and will include more youth in the risk pool than will ever go on to experience risk 
outcomes. The more precision one desires in an estimate of risk, the more difficult 
the task becomes, both because antecedents and markers are never perfect predictors 
and because the quality of the data gets significantly worse (or nonexistent) as the 
variables are more closely connected to problem behaviors or risk outcomes. 



TRADITIONAL SERVICES FOR AT-RISK YOUTH 

Traditional services for at-risk youth often address only a single risk marker or 
outcome such as adolescent pregnancy and parenting, substance abuse, delinquency, 
or school failure. Here we present a brief overview of the range of such programs, to 
establish the context in which we will consider the need for and potential contribution 
of service integration efforts for young adolescents. 

Several key parameters determine the current state of service provision for at- 
risk youth. Although many "traditional" programs rely on categorical Federal funding 
sources for at least part of their support, most Federal health- related spending for 
services to adolescents are entitlements rather than discretionary programs (Office of 
Technology Assessment 1991). In fact, Federal spending for adolescents under 
Medicaid dwarfs spending for adolescents by the National Institutes of Health, the 
Centers for Disease Control, the Alcohol, Drug Abuse and Mental Health 
Administration, and other DHHS agencies combined (Office of Technology Assessment 
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1991). Additionally, the bulk of discretionary funding is in the form of block grants to 
states; no Federal mandate requires these block grants to support youth services, and 
states often do not allocate dollars for youth-targeted programs. Of other 
discretionary spending, most programs address specific categories of youth problems, 
typically school problems, adolescent sexuality, drug use, and to some extent, 
delinquency (Office of Technology Assessment 1991). It is very common for local 
government and private foundation funding to follow the Federal model and focus on 
remedies for specific problems rather than addressing the overall problems of at-risk 
youth. 

There are exceptions to the "categorical" straitjacket, of course, in the form of 
organizations that have always had youth development as their focus (e.g.. Big 
Brothers/Sisters, Girls, Inc. and Boys/Girls Clubs. Many youth-serving organizations 
operate at the neighborhood level and follow a prevention-oriented approach that is 
gaining increasing recognition (Quinn 1992). 

The nature of Federal support controls the structures of the existing service 
system for adolescents. As a result of service system features, most programs for 
adolescents focus on treatment rather than prevention. To receive services from 
categorical Federal programs, youth must meet eligibility guidelines, which usually 
require evidence of serious disturbance or dysfunction. However, as we have already 
argued, for 10- to 15-year-olds it is more appropriate to define high risk by a 
combination of risk antecedents and markers, rather than expecting problem 
behaviors or risk outcomes. This implies that prevention rather than treatment 
services should be the primary means of serving this population (Dryfoos 1990). We 
look at traditional programs with an eye on their ability to offer appropriate services to 
early adolescents. 

Below we highlight the weaknesses and constraints of the present system, 
particularly its failure to provide a comprehensive and coordinated approach to the 
many problems of at-risk early adolescents. 
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Definition of Prevention Strategies 

One of the key distinctions between prevention and treatment is that prevention 
efforts target the processes that lead to dysfunctional states, rather than the states 
themselves (Lorion, Price, and Eaton 1989). Conversely, treatment services are 
intended to cure or ameliorate the effects of a problem or condition once it has 
occurred (Office of Technology Assessment 1991). The generally accepted view of 
prevention as comprising a triad of efforts-primary, secondary and tertiary— is derived 
from the public health arena and was proposed by Caplan (as cited in Lorion, Price, 
and Eaton 1989). Primary prevention involves efforts to reduce the incidence of new 
cases in the population and avoid the onset of a problem. Secondary prevention tries 
to reduce prevalence, that is, the total number of cases in the population. Secondary 
prevention efforts involve screening the target population to detect those most likely to 
continue the dysfunction and then intervening early (Lorion, Price and Eaton, 1989). 
Finally, tertiary prevention efforts seek to minimize the long-term and secondary 
consequences of a disorder among those already "diagnosed" as having the particular 
problem state. 

Cross-Cutting Issues for Traditional Youth Services 

Although each single-issue program confronts its own set of issues, Dryfoos 
(1990) has identified a number of elements common to successful programs 
regardless of the problem area they address. These include intensive individualized 
attention, community-wide multi-agency collaborative approaches, early identification 
and intervention, including school-based activities, administration of school programs 
by agencies outside of schools, including programs outside of schools, and 
arrangements for training in social skills. 

Traditional, Single-Issue Prevention and Treatment Strategies 

This section is organized around specific risk outcomes of adolescence, primarily 
because the nature of the existing service system is structured in this way. As we 



shall see, this may not be the most effective or efficient method for serving at-risk 
youth, particularly the younger adolescents (10- to 15-year-olds). This chapter 
summarizes common elements of many programs, the reader interested in details 
about the programs should refer to Resnick et al. (1992). 



School Failure and Dropout 

Programs generally focus on preventing school failure for younger adolescents 
and preventing dropout among older adolescents. Generally, programs that aim to 
prevent school failure deal with improving the quality of education in order to improve 
the achievement of all students. Dropout prevention programs include school-based 
as well as community-based interventions. School-based interventions include special 
curricula, structural reorganizations of schools, special services and counseling 
interventions, alternative schools, and multi-component programs. Community-based 
programs involve school-community and school-business partnerships to motivate 
students for higher achievement and to keep children in school longer. 

Most preventive programs strive to provide individualized attention, yet few have 
the resources to provide supportive services. Several programs include family 
components, and research supports the importance of parental involvement in 
improving student achievement scores, school attendance, motivation, and in 
assisting young adolescents to resist peer pressure (Mazur and Thureau, 1990). 

While dropout prevention programs try to bolster parental involvement in the 
educational experiences of 10- to 15-year-olds, few programs address the associated 
problem behaviors. In addition, little evidence exists to show that traditional dropout 
prevention programs are effective. 

In a review of all school failure and dropout prevention programs, Dryfoos (1990) 
listed the key elements of successful programs, including: a) variety and flexibility in 
approaches: b) early intervention: c) identification and continued monitoring of high- 
risk students from K through grade 12; d) small size of school and classes; e) 
individualized attention and instruction: f) program autonomy and clear lines of 
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responsibility for program planning and implementation; g) committed teachers who 
have high expectations for their students and are sensitive to cultural diversity; h) 
strong vocational components to strengthen the link between learning and working; i) 
intensive, sustained counseling for high-risk students, including counseling, social, 
and health services on-site; j) positive, safe school climate with a "family" atmosphere; 
and k) integration between community and school in planning of programs. No 
consensus exists on the benefits of several preventive interventions intended to reduce 
school failure and dropout (Dryfoos 1990). These interventions include: alternative 
schools, supplemental programs authorized by Chapter 1 of Title I of the Elementary 
and Secondaiy Education Act of 1965, extending the school day or school year, 
financial incentives for school completion, and school choice. 

Adolescent Pregnancy 

Programs aimed at preventing adolescent pregnancy in school settings use 
classroom curricula and school-based clinics; those in community settings use peer 
mentoring projects, family planning clinics, and youth-serving agencies. Most experts 
agree that family involvement in prevention programs for 10- to 15-year-olds is 
extremely important. Within this age group, a youth’s values and beliefs are largely 
defined by the attitudes and behaviors they learn at home. Parental involvement 
components of pregnancy prevention programs encourage parent-child 
communication about sex-related issues. Although one study found that increased 
parent-child communication about sexuality issues may not lead to a reduction in 
sexual activity or unintended pregnancy (Jorgensen 1991), most programs report the 
increase in intergenerational communication as a program benefit. 

A number of general concepts appear to guide the most successful of these 
prevention efforts, including: a) early intervention, no later than the middle school 
years; b) a package of services that includes both life-option and leadership 
development components; c) public commitment by local officials and community 
leaders to the prevention goal; d) the inclusion of males; e) services that maintain the 



youth’s confidentiality and privacy; f) better outreach, improved access to 
contraception, and effective follow-up of contraceptive users; g) improved access to 
pregnancy testing, counseling, and abortion services; h) involvement of parents 
wherever possible (not only in family life education approaches, but also in social 
skills training approaches); i) locating prevention efforts in the schools; j) 
implementing new curricula that include attention to social skills and life planning 
(which in turn requires better teacher training); k) involvement of outside community 
organizations in partnership with the schools, 1) availability of crisis intervention and 
referral mechanisms; and finally m) an array of comprehensive services for high risk 
youth, including alternative schools, preparation for employment, job placement, and 
case management. Below, we summarize Dryfoos’ review of prevention programs for 
substance abuse and delinquency. 



Substance Abuse 

The literature on substance abuse prevention is "extensive, diverse, uneven, and 
difficult to summarize" (Dryfoos 1990). Few studies consider all types of substance 
abuse, including cigarette smoking, alcohol abuse, and abuse of other drugs. There is 
also substantial disagreement among researchers about whether prevention programs 
should try to promote abstinence or responsible behavior and decision-making, and 
whether the prevention efforts should be directed solely at substance use behavior 
and decisions or should also include attention to ameliorating the effects of risk 
antecedents such as family dysfunction or neighborhood influences (Dryfoos 1990). 
Finally, some prevention approaches rely on enforcing restrictive laws to reduce use, 
rather than on programs that try to change the risk factors in the lives of potential 
users. 

According to Dryfoos, the elements of successful substance abuse prevention 
programs include: a) an approach that views substance abuse in a broad social and 
environmental context; b) comprehensive, community-wide prevention efforts directed 
at all major institutions; c) multiple interventions; d) schools (particularly middle 
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schools) as the central agency for locating prevention programs; e) a long-term 
approach starting with young children and age-appropriate components; f) teacher 
training; g) a full-time substance abuse coordinator; h) social skills training, including 
coping and resistance models; i) peer-led programs; and j) individualized attention and 
intensive counseling. Some unresolved issues in this field include the effectiveness of 
mass-marketed, packaged curricula; targeting programs only at high risk students; 
and the current lack of programs dealing with the "new drugs" such as crack. 

Dryfoos (1990) noted that "the history of substance abuse prevention is replete 
with failed models." According to her review, the programs which appear least likely 
to succeed include those that focus narrowly on only one avenue of change. Avenues 
that have, by themselves, failed to produce results include information or cognitive 
approaches, attitude change, self-esteem enhancement or affective methods, scare 
tactics, and "Just Say No to Drugs" media campaigns. 

Delinquency 

There are few examples of traditional delinquency prevention programs, primarily 
because issues of adolescent crime are most often addressed in dropout prevention or 
violence prevention programs. Furthermore, research to date suggests that efforts to 
prevent delinquency among adolescents have been largely unsuccessful, and one 
expert recommended that traditional delinquency prevention efforts be abandoned. 
Dryfoos (1990) summarized the literature on programs that do not work and listed 
among these preventive casework, group counseling, pharmacological interventions, 
work experience, vocational education, probation officers, the use of traditional street 
corner workers, social area or neighborhood projects, and "scaring straight" efforts. 
There appeared to be some consensus around what programs are likely to be effective, 
including; a) broad-based goals that go beyond delinquency prevention; b) multiple 
components; c) early interventions, prior to adolescence; d) involvement of schools; e) 
direct efforts at institutional rather than individual change; 0 individual intensive 
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attention and personalized planning; g) good quality control over treatment integrity; 
and h) long-term follow-up and continuity of service. 

Limitations of Traditional Programs 

Over the years, traditional single-focus programs have encountered a number of 
limitations. First, these programs have often recognized that the social and 
supportive services they offer do not address some of the most pressing needs of their 
clients. Second, they have found that when they identify a need they cannot meet 
with program resources, it is sometimes difficult for other agencies in the community 
to help their clients. The problem may be eligibility— the client is not poor enough, or 
not officially part of the target population of the agency with the resources, or not the 
right age, or does not have the right address. Or the problem may be availability— 
there are only so many day care slots, housing vouchers, and so on. Or the problem 
may be accessibility or appropriateness— the services are not hospitable to youth, or 
cannot be reached by public transportation, or are not open at the right hours or on 
the right, days. 

Frustration with these barriers sets the stage for programs to: 1) to try to expand 
their own services to cover the most important gaps and 2) begin negotiations with 
referral agencies to try to smooth the process of getting services to clients across 
agencies. The former reflects program efforts to become more comprehensive under a 
single roof; the latter reflects efforts to achieve more formal or informal integration of 
the service agency network within a community to assure compliance service delivery 
when needed. The remainder of this chapter addresses issues posed by service 
integration efforts. 



ISSUES IN SERVICE INTEGRATION 

Integrated service models to deliver comprehensive services to youth and their 
families through collaboration, cooperation, and coordination of efforts have received 
increased attention recently in response to the many and varied service needs of 
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youth, and the frustrations encountered by traditional single-problem approaches to 
service deliveiy. Calls for service integration (SI) have come from various sources, 
using varying terminology and different meanings for the same terms. Below we 
define the meaning of the terms we use here for the sake of clarity and not to imply 
the endorsement of one approach or viewpoint over another. 

Attempts to serve at-risk youth have encountered all of the same service system 
issues that plague current efforts in the United States to serve any target population 
in a way that meets all of their needs. These issues include: 

• Comprehensiveness- the existence in the community service system, or in the 
intake agency itself, of the full range of services needed to address the needs of 
the target population. 

• Service Levels — enough of the appropriate services to assure that eveiyone in all 
the probable target populations in the community could use the service if 
necessary. 

• Service lntegratlon -the ability of the agency through which a member of the 
target population enters the system to assure that its clients receive the services 
they need, regardless of which community agency offers the services, because the 
intake agency has developed the necessaiy relationships to assure access with 
other service agencies. 

It is theoretically possible to have a comprehensive system that is not integrated, 
as when a single agency (usually private) has the resources to provide everything its 
clients need. It is also possible to have an integrated system that is not 
comprehensive, as when an agency serving at-risk youth and their families only 
negotiates arrangements with those services it has found to meet the most common 
needs of its clients, such as income maintenance, child care, recreation, and 
education services. It may not, however, have similar well-established arrangements 
with agencies with which it does not interact so routinely. 

Finally, it is possible for a given community to have the entire array of service 
types, and to have regularized inter-agency arrangements for assuring that clients can 
access the services, and still not have enough of some services to serve all the people 
who need them. This last circumstance probably characterizes most communities, 
and is a limiting condition for the possible impact of any SI effort. But a systematic SI 
effort can make the need for more services so apparent that legislatures and other 





40 



funders may respond by supporting service expansions where need has been 
documented and a structure is in place to assure that the additional services will be 
well used. 

When we began this investigation we used service integration (SI) to refer to 
procedures and structures that help several service agencies coordinate their efforts to 
address the full range of service needs presented by youth and families in an efficient 
and holistic manner. While relatively few existing systems actually meet all the 
elements of an ideal SI model, we can propose several key characteristics that should 
be present in such an SI system for at-risk youth. These include: 

• An approach to helping at-risk youth that sees each youth for himself or 
herself, and also sees the youth as part of a family, a neighborhood, and a 
co mmuni ty that may in turn be influenced to reduce the risk that a youth will 
participate in problem behaviors or experience risk outcomes. 

• A comprehensive, individualized assessment at or near the point of intake, 
that is conducted for each youth and family, to identify the full range of his or 
her individual and family service needs. 

• A coordinated service plan that, based on the needs identified, is developed to 
ensure that all needs are addressed in an efficient fashion by the program(s) best 
suited for the task. 

• Institutionalized inter-agency linkages that ensure that service referrals result 
in actual service delivery. This may entail an inter-agency case management 
function, co-location of services at a single site, and/or sharing of other 
resources among programs. 

• Follow up on service referrals, to ensure that services are delivered in an 
appropriate manner and that the program coordination structures are 
functioning effectively. 



In reality, relatively few programs meet these formal criteria for SI. However, a 
considerably larger number of programs meet the spirit of the assessment, service 
plan and follow-up criteria through intimate and regular connections with young 
clients and their families. They also meet the inter-agency linkage criterion through 
informal but effective arrangements with other service agencies, which they have 
developed over the years of working to meet their clients’ needs. Their "failure" is 
more likely to be with documentation than with performance in getting services to 
clients. After visiting a number of programs, we want to propose another aspect of 
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service integration: the ability of a program to fill the gaps in service identified 
through the joint efforts of community agencies. The resulting program may be the 
agency to which everyone else refers youth because the referring agencies cannot meet 
all the needs of these clients. The program has developed components cooperatively 
with the referring agencies to meet these identified needs. The formal inter-agency 
arrangements are for referral into the program rather than for referral out from the 
program. Once the youth reaches the program, it may be that not much by way of 
multi-agency service use occurs— but it does not need to. 

We think programs of this type deserve to be called an SI program or, even 
better, an SI community: the program is the glue that holds the system together. 

Some of the programs described later in this report are of this type. 



History of Service Integration 

Interest in and efforts at youth-centered service integration in both school and 
community settings have ebbed and flowed over the years, with varying degrees of 
commitment and success (Tyack, 1992). Most public programs aimed at enhancing 
conditions for youth and families over the last half-century have been focused on only 
one or a few problems from the perspective of a single service system such as welfare 
or criminal justice. These traditional programs often dealt only with the youth, rather 
than addressing multiple needs of their families and their neighborhoods (Ginzberg et 
al. 1988). 

The 1960s saw a marked reawakening of interest in and experimentation with 
both comprehensive and integrated service delivery systems. The Federal government 
invested a good deal in human services programs as part of the 1960s "War on 
Poverty." A very important aspect for SI of the programs from this era is that they 
were designed to be developed from the bottom up to meet the needs of specific 
communities. Funding structures deliberately bypassed state government agencies, 
which were seen as unable to respond to local community needs. National programs 
such as Neighborhood Service Centers, family planning agencies, and Head Start had 
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a decidedly community orientation. Some were able to evolve into comprehensive 
programs, and some incorporated some type of SI structure. Although these 
programs did not eliminate poverty, many did succeed in pioneering a community- 
based approach to services, flexibility in meeting local needs, and attention to the 
larger context of client problems in family, neighborhood, and community. 

The 1970s saw a more modest approach to such efforts (Edelman and Radin, 
1991; Kusserow, 1991). Service integration efforts of the 1970s focused more on 
coordination of categorical programs at the Federal level and funding of smaller 
demonstration projects at the community lev 1 than on large-scale system reform. By 
the late 1970s and through the 1980s, the opportunity for SI initiatives devolved 
largely to state and local governments. Block granting of Federal social services 
funding in 1975 (Title XX) and of 30 additional categorical programs in 1981 
eliminated many program rules and technically gave states greater flexibility to 
provide services out of a larger pool of resources than any categorical program had 
previously enjoyed. However, the concurrent funding cuts in the 1981 restructuring 
severely curtailed state efforts to innovate. Simply maintaining service levels was hard 
enough. 

The recent renewed interest in SI is attributable to several factors. There has 
been a renewed appreciation of how ineffective it can be to deliver services in a 
fragmented, problem-oriented fashion. In addition, some integrated approaches to 
service delivery have shown positive results and served as models for this type of 
approach (Berreuta-Clement et al. 1984). Advances in research on adolescent 
development and ecological and family systems theories (e.g., Bronfenbrenner 1979) 
have also helped revitalize interest in service delivery systems that respond to both 
youth and environment using a more integrated, holistic approach. So has the 
concerted effort to address the complex problem of long-term welfare dependency, 
culminating in the Family Support Act of 1988 (FSA). The FSA recognizes the need to 
address a wide variety of issues a family may face in trying to achieve self-sufficiency, 
and directs states to develop systems to meet many family needs. Many of the family 
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needs recognized by the FSA are the same ones that youth-serving agencies try to help 
families handle. Finally, spartan fiscal conditions on the state and Federal levels have 
created an impetus to service integration (Corriea 1992). 

Barriers to Service Integration 

SI efforts face many barriers, including professional training and orientation, 
administrative procedures, eligibility rules, and the categorical nature of funding. 
Service agency staff are typically trained in rather narrow, specialized traditions such 
as mental health or criminal justice services, and may not feel comfortable dealing 
with other issues or working within an inter-agency framework. 

Administrative and bureaucratic procedures often obstruct SI efforts, agencies 
may insist on following their own intake and case processing procedures, and 
confidentiality requirements may limit their ability to share information about clients 
with an SI team. Categorical funding from government agencies, foundations, or other 
institutions also perpetuates single-issue programs. As long as legislatures and 
funders structure programs to address specific issue areas, single-issue programs will 
continue to provide services and have difficulty making their services available to 
populations not specified by their mandate. 

Another barrier is that categorical programs usually focus on problems and tend 
to support short-term efforts. Programs that try to solve problems quickly and then 
close the case are not likely to meet the needs of youth: first, they are not geared 
toward preventive interventions: second, they often have little staying power. 

Access to services due to the fragmented nature of single-issue service delivery 
was identified by the Office of Technology Assessment (1991) as a critical problem for 
adolescents. Adolescents most likely to have access problems are those who: lack any 
or adequate health insurance: are unaware of services or feel intimidated by public 
agencies: need parental consent to receive services but are in potential conflict with 
their parents: are homeless or incarcerated in juvenile justice facilities: live in rural 
areas without services: and are members of a minority group. In addition to 



confronting access barriers, youth cannot always get appropriate treatment services. 
Even if adolescents do gain access, the services may not be suited to their 
developmental level and their level of real-world experience. 

The barriers to SI discussed so far pertain to government agencies. But most 
youth are not likely to approach government agencies on their own. Their entry to the 
service system will probably be through nonprofit community or youth development 
agencies and neighborhood programs. Pittman and Cahill (1992) report that youth 
tend to seek services and maintain a relationship with a service organization when it 
has a distinctly youth focus, many other young clients/users/members, a 
"membership" orientation (youth can stay with the program for a long time), staff who 
enjoy working with youth, and many attractive activities (rather than a strictly 
problem/service focus). Therefore SI efforts may need to start where the youth go, 
and work with those agencies to help them gain access to the more formal systems for 
their clients when the need arises. 

Lessons Learned 

Kusserow (1991) summarizes the lessons for the future learned from the past 
twenty years of SI efforts: 

• A SI strategy likely to generate more near-term success should focus on well- 
defined target groups and pursue reform primarily within categorical program 
areas. 

• Even a target-group, categorical-program approach, however, is likely to require 
some degree of central authority and flexible funding to generate and sustain 
more integrated service delivery. 

• A funding source granting an organization some authority and flexible funding 
for promoting SI should hold it accountable for defining and measuring expected 
outcomes. 

• The cultivation and maintenance of networks of individuals engaged in SI efforts 
are vital to the success of these efforts. 



Approaches to Service Integration 

Given the renewed interest in SI, it is important to highlight some issues that 
have emerged from earlier experiences with SI efforts. These include their mission, 
their underlying views of youth and their service needs, and the nature of the service 
delivery network. 

Mission 

One reason it is difficult to describe SI approaches as a whole is that different 
advocates and different programs bring different missions to SI. Unless we know 
what a program is trying to accomplish with SI. it is hard to know what success 
should look like. 

Some see SI as enhancing a service mission by delivering more services or more 
appropriate services or more complete services, or by delivering services faster and 
with less hassle for the client. Some SI proponents may have agency-oriented goals, 
such as saving money by using integrated application procedures or reducing the time 
that case managers spend negotiating separate delivery systems. But another 
mission-one apparently shared by the best youth-serving agencies (Pittman and 
Cahill 1992)— is attracting youth to self-enhancing activities. 

Rather than simply working to avoid risk, self-enhancing activities often involve 
older youth and family members, and give youth opportunities to solve their own 
problems by helping themselves, their family, and their community. Pittman and 
Cahill warn that this mission, which they consider paramount, usually gets lost in 
discussions that concentrate exclusively on service breadth and depth— which 
services? how many services? to whom? required or voluntary? required for everyone 
or only some? on-site or off? These questions, they say, "suggest that instrumental 
changes in the way services are delivered will result in improved outcomes for youth 
... the questions limit discussion to a technical dimension instead of including a focus 
on mission and outcomes ... the result is often an adding on or adjustment of current 
services" rather than engaging the whole community in goal-setting and program 



design. SI efforts may emerge as part of a program designed this way. but the 
measures of program success would certainly not be "services delivered" or "money 
saved" (Pittman and Cahill 1992). 

Perspectives on Youth. 

Perhaps most basic is the fundamental perspective one holds on youth and their 
need for services. A holistic approach values children and youth as people to be 
supported and nourished so they may become effective future workers, parents, and 
community members (Quinn 1992). This perspective underlies the use of 
comprehensive, individualized assessments of service needs and service planning, 
and the sense of respect for youth also encourages empowerment efforts by focusing 
on strengths, potential for exerting leadership, and potential for making contributions 
beneficial to others (Pittman and Cahill 1992). 

For preventive or ameliorative efforts to work well, they must address the causes 
underlying youths’ needs for services. Family dysfunction and the neighborhood 
context are two of the principle antecedents of problem behaviors and risk outcomes 
for youth. Programs desiring to make a real difference for youth should directly 
involve parents, other family members, older peers or role models, and the youth’s 
neighborhood friends and peer group in activities designed to reduce risk and promote 
healthy development (Pittman and Cahill 1992; Schorr and Schorr 1988). 



Partnerships 

A fundamental feature of SI is its emphasis on cooperation or partnerships 
among a wide variety of key agents or "players" (Dryfoos 1990; Hechinger 1992). 
Youth, their families, and other key individuals and organizations in the community 
can be instrumental in identifying service needs, in planning and implementing 
service programs to address them, and in proposing a program structure that will be 
most appealing and accessible to its target population. This is a first step in 
empowering youth and families. 
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All levels of public and private local service agencies must be involved to some 
degree, from top management to line workers. Other community groups such as 4-H 
Clubs and churches can provide technical assistance or volunteers, and occasionally 
help out with funding (Ledwith 1990). Richman, Wynn, and Costello (1991) describe 
an integrated service system for children based on collaborative arrangements among 
"primaiy" services (community organizations such as sports teams, parks, and 
museums) and "specialized" services (the more formalized health, education, and 
social service agencies) to address the needs of all the children in a community. 

Private foundations and philanthropic organizations can assist service integration 
efforts by giving financial support, technical assistance, or volunteer staff. For 
example, the Chicago Community Trust provides a steering committee and up to $30 
million over this decade to support the "Children, Youth, and Families Initiative"— 
aimed at creating a comprehensive, integrated, community-based service system to 
help Chicago families and their children. In addition, businesses can provide funding, 
management assistance, summer jobs, volunteers, and political support; the media 
can assist with public education and awareness efforts (Dryfoos 1992; Ledwith 1990). 

The central executive arm of local, state, and Federal governments can also help 
in a number of ways. Local leaders can assist by nurturing community and political 
support for SI, directing key agencies to cooperate, and developing local solutions to 
local problems. State governments can contribute by funding planning and 
implementation efforts, supplying technical and management assistance, helping to 
design and establish a management information system, and aiding the development 
of a common language, set of regulations, and administrative procedures for use by 
various service agencies (Melaville and Blank 1991; Quinn 1992). 

The Federal government has undertaken a variety of initiatives to support youth 
service integration. The Office of the Assistant Secretary for Planning and Evaluation 
(ASPE) in the Department of Health and Human Services (DHHS) has recently 
established the National Resource Center for Community-Based Service Integration to 
provide technical assistance, serve as a clearinghouse, and help establish inter-agency 
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linkages. ASPE is also collaborating with the Office of Educational Research and 
Improvement in the Department of Education to produce a guidebook on developing 
school-linked comprehensive services. 

In addition to providing support in this manner, ASPE also provides funding to 
plan and implement a number of comprehensive service integration efforts across the 
nation. The Council of Governors’ Policy Advisors’ Second Academy on Families and 
Children at Risk is a seven-state service integration planning and implementation 
effort co-funded with DHHS’ Administration for Children and Families and the Ford 
Foundation. Other more localized ASPE-sponsored programs include school-based 
service programs in Florida and California; community-based services in Georgia; and 
funds to support joint inter-agency planning in Ohio. 

In addition, the Presidential Empowerment Task Force’s Service Integration Work 
Group identified successful SI models and methods to improve inter-agency 
communication and coordination at the Federal level. The Task Force has also 
concerned itself with restructuring statutory and regulatory requirements to improve 
service access, coordination, and quality (Gerry and Certo 1992). 



Steps in Planning and Implementing 
Comprehensive, Integrated Services 

Below we present some of the major issues and alternative implementation 

strategies that should be considered when implementing SI. 



Defining Goals and Objectives 

As the first concrete step in the planning process, the partners involved should 
work toward agreement on a common set of goals and objectives (Center for the 
Future of Children 1992). To the extent possible, long-term commitment to the 
integration effort should be built in from the planning stage. One effective method for 
encouraging long-term commitment is through an independent inter-agency advisory 
group with a revolving chair, to help minimize turf battles and forge a common 
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purpose for the variety of service integration partners. Another method involves 
diversion of a portion of each partner’s funds to support the integration effort, so each 
partner has an important stake in assuring success of the integrated approach. 

The program’s goals should be based on a local community needs assessment 
and an assessment of services already available, whether formal or informal. If the 
full range of stakeholders is included in the planning process, knowledge of service 
needs and adequacy of existing services should be included. Efforts should be made 
to solicit input and build support from as many of the partners as possible. Outside 
consultants can also be brought in to share their expertise (Corriea 1992). 

Identifying the Target Population 

Who should the newly integrated services be designed to help? Unless the target 
population is clear, it will not be obvious what services and other activities should be 
incorporated into the effort. Whether services should be offered to all youth and 
families in the community, or only to those considered at highest risk, is an important 
policy question for local partners to address (Levy and Shepardson 1992). Once a 
youth or family enters a program involved in SI, agencies should have sufficient 
knowledge of services available, inter-agency cooperation, and flexibility to ensure that 
all of their service needs are identified and addressed. Some authorities maintain that 
services should be concentrated on those who are most at risk; others argue that this 
approach would stigmatize program participants, and that all children could benefit 
from enrichment efforts (Dryfoos 1990). 

There is no definitive profile of youth or families who need SI. However, families 
involved in alcohol or drug treatment may be prime candidates for activities and 
services to improve their support for their children. Families involved with child 
welfare due to reports of abuse or neglect clearly need help in supportive parenting. 
Equally important is identifying families who have none of these problems but who 
struggle to raise their children with little money and few resources in neighborhoods 
that pose a constant threat to their children’s future. 



The Office of Technology Assessment (1991) concluded that adolescents who are 
not currently being served by the myriad of prevention and treatment programs are 
those "with, or at risk of multiple problems, who almost inevitably face gaps among 
service systems" (p. 1-30). Adolescents most likely to encounter service gaps are those 
with substance abuse and mental health problems, adolescents adjudicated as 
delinquent but who probably have multiple health problems, homeless adolescents, 
and adolescents failing or misbehaving in school who are also likely to become 
pregnant, delinquent, and/or drop out of school. 

If a program targets 10- to 15-year-olds and their families, a different array of 
activities and services are likely to be needed than if an older adolescent population 
were the target. For the younger group, prevention activities involving recreation, 
community service, self-esteem and competence building, compensatory educational 
efforts and similar activities will be primary, with treatment services on reserve and 
accessible if needed. Older youth may need a stronger mix of treatment services to 
help them stop participating in problem behaviors, as well as the supportive 
developmental services offered to younger teens. 



Identifying the Services to be Offered 

A comprehensive approach involves a child- and family-centered orientation 
approach in which the range of each family’s service needs are identified and services 
are planned and delivered to address their unique situation. This contrasts with a 
problem-centered approach, in which an agency addresses only the specific problems 
it is prepared to handle itself. A comprehensive approach requires considerable 
variety in the breadth and depth of services available and flexibility in service delivery. 
It is always important to remember, however, that "comprehensive" and "integrated" 
are not identical. The point of developing a service structure is not to assemble the 
largest numbt r of services, but to help youth and their families. Successful youth 
serv ce programs are marked by their common emphasis on client empowerment 
rather than on narrowly defined "services" from public agencies (Pittman and Cahill 
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1992). SI comes into the mix only in Pittman and Cahill’s final program 
characteristic-community "clout." the ability to get clients the services they need that 
come from other agencies in the community. 

The type of services to be offered, including outreach, public education, primary 
and/or secondary prevention, intervention, and advocacy, needs to be decided on the 
basis of local needs and resources. With young adolescents, primary and secondary 
prevention is likely to be a major focus. 

The breadth of services is another issue. In one view, a minimum of two specific 
types of services in each of the three broad categories of education, health, and social 
services should be offered for the program to be considered truly comprehensive 
(Morrill and Gerry 1990). Others argue that basic life skills such as critical thinking, 
problem-solving, and decision-making, social skills such as constructive 
assertiveness, and the use of social support systems should be the program’s focus 
(Hechinger 1992). 

The intensity of services should also be considered. The service programs should 
be flexible enough to respond to clients who may require more frequent services or 
services that address the relevant issues in more detail. 

For the target population of young adolescents, there is also some question about 
the best way to provide comprehensive services. The more a program emphasizes 
prevention, the more it may focus on developing self-esteem and positive life skills, 
resisting peer pressure to participate in risky behaviors, and fostering a belief that 
youth can have a positive and productive future as an adult. Programs may promote 
these goals through emotionally supportive role-modeling from mentors or big 
brothers/sisters. A comprehensive program in this context would assure that the 
mentor has access to someone in a case management role when it becomes apparent 
that a youth needs a particular type of help. In contrast, a program that involves 
heavy up-front assessment and case management may be more appropriate for the 
small proportion of 10- to 15-year-olds who need massive early intervention. 
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Mechanisms for Service Delivery 

The way in which services are coordinated is important. Clients may have a 
service agency contact with whom they maintain an ongoing, supportive relationship. 
When this contact person functions more as a mentor, counselor, or group worker 
than as a case manager, this individual needs access to someone who can arrange 
needed services and follow up on referrals. 

Case management— a key issue— is essentially a method of placing responsibility 
for service planning, coordination of service delivery, and follow up on an individual or 
inter-agency team. The case manager or team works with youth and their families to 
determine service needs, provide inter-agency linkages, and monitor service delivery 
and outcomes (Melaville and Blank 1991). Effective case management requires 
relatively smaller caseloads as the needs of clients increase. Intensity of services 
offered should be determined at least in part by the youth and family’s ability and 
motivation to work with the system. The procedures established should be flexible 
enough to respond to each youth and family’s unique circumstances. 



Service Location 

Integrated services can be delivered through school-based or school-linked sites, 
in community sites such as churches or community centers, through mobile 
arrangements, and/or by home visits (Mathtech, unpublished manuscript). We are 
unlikely to find a universally applicable program model. In all likelihood the location 
of an SI effort will depend on which agency or organization has an interested, 
committed, and dynamic person willing to take the lead in developing and running the 
program. Another important factor is the site’s acceptance within the community. 
Occasionally a local agency may get involved in SI because some funding source has 
invited its participation. Such invitations are most likely to be accepted when there is 
local leadership to carry the program. 

Services are typically based in either school or community sites. School-based 
programs have the potential to reach large numbers of youth, and have a well- 
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established organizational structure and niche in the community, but may not be as 
accessible to families or to youth and families who are alienated from the educational 
system, such as high risk dropout youth. They may also further stress an 
overburdened educational system (Chaskin and Richman 1992), may be restricted as 
to which services they can provide (e.g., family planning services), and may be 
constrained by rigid organizational rules. Community-based programs may avoid 
these problems but face issues of access for youth and families and high-crime and 
gang-infested neighborhoods. 

When the school-based program under consideration is an adolescent health clinic 
a number of special barriers arise. These include lack of trained personnel, and 
community resistance to the role these clinics may play in sex education and in 
contraceptive counseling and distribution (Office of Technology Assessment 1991). 

Debates also occur about the appropriate balance of services between on-site and 
off-site locations. Some programs aspire to on-site "one-stop-shopping," while others 
function as a link between clients and a very broad spectrum of services— none of 
which is offered on site. The debate about service concentration usually involves the 
relative benefits of ease of access versus learning to negotiate the systems oneself. 
Most programs fall somewhere between these two extremes. A community just 
beginning to develop SI should consider this issue. 




Administrative Factors 

To be a credible model of service integrati i, the agencies involved should have 
institutionalized linkages that establish the mechanisms for sharing resources. These 
mechanisms may include co-locating in a single facility; sharing staff, financial 
resources, and/or information; and agreeing to provide services to referred people. 

An agency that provides needs assessments, service referrals, and referral follow 
ups must be able to give referral agencies the information it has about a client’s 
needs. Many agencies have confidentiality policies that prohibit the disclosure of 
client information between service agencies, and sometimes even within different 
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divisions of a single agency. For SI to work, agencies must find ways to adjust these 
confidentiality policies and still protect sensitive information about clients. Gaining 
the informed consent of clients to share information with agency personnel who will 
be providing the referral service is one approach that has worked in some places. But 
even this may require formal legal or rule changes. 

Staffing Issues 

It is important that staff be recruited and trained very carefully, whether they are 
paid or unpaid (Primm Brown 1992). Staff should be selected on the basis of their 
ability to establish trusting, respectful relationships with youth and families, their 
ability to span professional boundaries and specializations to address clients’ needs, 
and their ability to work with the system, whatever their type or level of professional 
training (Sonenstein et al. 1991). 

Diversity issues must also be considered in staffing programs (Corriea 1992). If 
at all possible, staff should reflect the racial, ethnic, age, and gender make-up of the 
program’s clientele. At an absolute minimum staff should have a demonstrated 
sensitivity to issues of racial, ethnic, and gender diversity, preferably through earlier 
work experience with populations similar to those expected to use the program. 

Staff support for the integration model and willingness to adopt new roles are 
crucial at all levels. Strong positive leadership is usually critical; neutrality is not 
good enough to shepherd a new program to successful implementation. 

Staff at all levels should be trained to work effectively within an integrated model. 
Training should be sensitive to the concerns of staff experienced in non-integrated 
service settings—concems such as "turf' issues, professional orientations and jargons, 
and issues staff may feel unprepared to deal with. 



Funding Issues 

Categorical funding streams established by Federal and state authorities are a 
major impediment to SI. Procedures for documenting the use of categorical funds are 





often prohibitively burdensome for small programs trying to provide many different 
services. Different program rules and reporting requirements may demand a level of 
administrative support that many programs simply cannot provide, and which the 
categorical funds do not support. Whatever the type of funding, insufficient resources 
induce competitiveness between service programs and undermine collaborative efforts 
(Farrow and Joe 1992). 

For SI to work best, funding should be flexible. Federal and state funding 
sources should be redesigned to blend together funds from multiple sources that 
historically have rigid categorical boundaries, to provide adequate and coherent 
funding for service programs that address multiple areas of need (Kirst, 1991). 
However, this is unlikely to happen. Even where system change has been a primary 
component of demonstrations with significant funding to support it, as in the Robert 
Wood Johnson Foundation programs for the severely mentally ill or the Annie E. 

Casey Foundation New Futures dropout prevention projects, only modest system 
change has been achieved at best. Since SI efforts do not invest anything 
approaching the level of resources in producing system change that characterized 
these demonstrations, it is unrealistic to expect much in this regard from SI efforts. 

Private funding is also available but not usually in sufficient amounts to serve as 
single-source funding for an entire integration effort. While some service integration 
efforts have successfully combined public and private funds to support widely 
respected service programs (e.g.. New Beginnings in San Diego), such success is not 
always the case. The need to match funds from various sources that may be 
concerned with different issues may sometimes result in scattershot, funding-driven 
programming, as well as an excessive administrative and development burden 
(Melaville and Blank 1991). 

One promising approach to increasing SI among already functioning programs is 
using limited new funding to support core integration functions. This effort could be 
matched by diverting some existing funds to support additional integration efforts and 
using other existing funds to support regular service delivery. Kentucky’s Family 
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Resource and Youth Service Centers, to be implemented in approximately 1,200 
schools across the state by 1995, is currently using such a financing plan. Its future 
funding base will be partly determined by the results of this approach. 

Evaluation 

There is a lack of valid and reliable evaluation results that test the effectiveness 
of programs and identify those program components that appear to contribute to 
program success. Experts cite a lack of funding as a major barrier to evaluation 
efforts, since most categorical programs consider service delivery the only eligible 
expenditure. Most serious evaluations are funded either by Federal government 
programs or by foundations, and often involve special demonstration efforts rather 
than "normal" programs operating in a variety of environments. 

Experience has shown that programs that look good as demonstrations often are 
diluted upon replication. This phenomenon suggests that evaluation results are used 
to justify program dissemination or replication, but are not reviewed in enough detail 
to assure that critical aspects of programs actually appear in replication. Dryfoos 
(1990) concludes that evaluation results are rarely used to make decisions about 
continued program structure or funding, especially for programs that are mass- 
marketed and packaged for schools and teachers. 

In order for evaluation to be satisfying for the program and influential in shaping 
its future, evaluators must have extensive early collaboration with program personnel 
so the measures used are meaningful and cooperation with the evaluation is high. 

Impact information should be tied to youth and family outcomes rather than 
simply services delivered. Outcomes should be realistically identified for established 
programs, and outcome information should come from a variety of sources, including 
program clients. Where possible, the most effective program characteristics or service 
delivery methods should be identified, to aid in further program refinement and 
assessment of program replicability (Morrill and Gerry 1990). 
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Information on cost effectiveness is crucially needed (Merrill and Gerry 1990). 
Data establishing how much money integrated services can save from participating 
and other agencies’ budgets, and when programs can expect to realize the cost 
savings, would be very useful in developing and evaluating funding requests. 

Institutionalizing Change 

A long-term SI issue is whether any changes created by SI in the component 
agencies’ functioning and interrelationships become institutionalized and take on a 
life of their own. Kusserow (1991), summarizing twenty years of SI activities, notes 
that "SI efforts have been instrumental in making human services more accessible to 
clients and more responsive to their needs. Over the long term, however, SI efforts 
appear to have had little institutional impact on a highly fragmented human services 
system." His list of major barriers to system change echoes issues discussed earlier in 
this chapter. 

• The size and complexity of the human services system; 

• Professionalization, specialization, and bureaucratization; 

• Limited influence of integrators; 

• Weak constituency for service integration; 

• Funding limitations; and 

• Insufficient knowledge. 

It is very important that service integration efforts rest on more than seed 
funding and strong personalities or leadership. Such factors are likely to be 
transitory. A program depending on these factors is likely to collapse when the 
funding expires and the individuals depart. Pooling at least a portion of each agency’s 
core funding to support integration activities is a systemic change that can be crucial 
in assuring the survival of the integrated service network. This practice may assure 
adequate resources to continue the integrated approach after start-up funding 
expires. It may also solidify the commitment of participating agencies by their very 
tangible stake in the SI structure (Melaville and Blank 1991). 

Where post-demonstration funding is inadequate to sustain the integrated 
approach, the availability of evaluation data documenting the innovative processes 



and beneficial outcomes resulting from the use of an integrated approach can be 
instrumental in securing continuation funding (Melaville and Blank 1991). 
Policymakers and (potential) funders can make better -informed decisions on how to 
allocate limited resources when information is available to document implementation 
procedures, service costs, and cost-savings. Even more desirable is information 
showing the impact of the integrated approach on program participants, component 
agencies, and the social service system. 

SUMMARY 

In this chapter we have examined common definitions of youth at risk, and 
developed a framework for thinking about the many disparate indicators and signs of 
risk. We organized our review of how many youth are involved in different risky 
situations according to our framework, looking first at prevalence of risk antecedents, 
then at system markers for risk, and finally at problem behaviors and risk outcomes. 

Following the review of prevalence information, we examined the most common 
approaches to helping youth at risk. These traditional programs are usually found 
within a single societal institution and frequently address a single problem. We then 
described some of the problems encountered by traditional single-focus programs that 
stem from the fact that their clients or users often had problems outside the focus of 
program expertise. The existence of these additional problems or issues often 
interfered with the program’s ability to help the youth address the problem for which 
he or she had come to the program. 

The difficulties encountered by traditional programs in accessing services outside 
their purview. >r their unwillingness to do so, has led to the current focus on 
comprehensive services and on service integration. We then discussed the goals of 
programs that try to provide comprehensive services or service integration, and the 
system resistances and barriers they often face. 



CHAPTER 3 



EVALUATION ISSUES FOR PROGRAMS SERVING TOUTH AT RISK 



This chapter briefly addresses key issues relevant to conducting evaluations in 
programs that try to deliver comprehensive services to at-risk youth in a service 
context of multi-agency collaboration and service integration. Many of the evaluation 
issues discussed in the chapter could receive extensive treatment as general issues in 
conducting evaluations of any service program. That is not the approach taken here. 
Instead, we summarize the evaluation issues particular to youth-serving programs 



and service integration (SI) efforts, including the possible obstacles to evaluating these 
programs. We do not refer extensively to specific evaluations of individual programs. 
Rather, we draw relatively heavily on a number of papers which critically review 
evaluations of youth-serving programs and of service integration efforts, as well as on 
our own experience in conducting evaluations for both types of programs. 



This chapter is organized into four major sections: 

• Evaluation issues specific to youth-serving programs , including: 

1 . Who should be considered a client: 

2. Differences in client risk levels that may affect services received and 
evaluation plans; 

3. The age range of interest and its implications for program configuration, in 
particular for documenting the program as delivered when activities rather 
than services are the program focus. 

• Evaluation issues specific to service integration efforts , including: 

4. Cross-agency documentation of service delivery: 

5. Identifying non-client outcomes of interest (e.g., system change): 

6. Identifying the benefits expected for clients from SI as differentiated from 
comprehensiveness. 

• Lessons learned from previous evaluations , including: 

7. Service/ program configurations likely and unlikely to make a difference: 

8. Evaluation approaches most likely to succeed (including who should do an 
evaluation and how to increase the willingness of programs to participate): 

9. Maintaining the program as evaluated once the evaluation (and 
presumably with it file "demonstration" level of funding) is over. 

• Evaluation issues specific to the types of sites selected for this study , including: 

10. Readiness for evaluation: 
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1 1. Identifying realistic outcomes and measures of those outcomes; 

12. Identifying appropriate comparison or control groups; 

13. Reducing attrition at follow-up. 

We end with a section on implications for preliminary site visits. 



EVALUATION ISSUES SPECIFIC TO 
YOUTH-SERVING PROGRAMS 

Youth-serving programs are structured in many ways, from those that focus 
completely on treatment for youth who have already exhibited serious risk outcomes 
to those that are completely focused on youth development and prevention in the 
most general way. A majority of the programs designed to work with younger 
adolescents— 10- to 15-year-olds— lean more in the direction of youth development and 
prevention than in the direction of treatment. The age range and the prevention 
orientation have implications for what programs offer youth, how they do it, and 
whether or not they include families and the community within their sphere of 
attempted influence. Each program variation affects how one would conduct an 
evaluation of the program. 

Defining the Participant and the Unit of Analysis 

Some programs have a clear way of knowing when someone becomes a client 
and when someone stops being a client. A formal intake procedure marks the entry 
point. Completing the full intervention marks exit from the program. However, many 
programs have some trouble deciding when someone has really become a client or 
when someone has stopped being a client. 

Defining a Client by Intake Status 

Most programs have some clearly identifiable intake procedures. A simple 
approach would be to define youth who have gone through these procedures as 
program clients; those who have not begun or completed the procedures are not 
clients. 



However, the process of attachment to a program can be vague. If a youth has 
one or two phone conversations with program staff or even pays the program one or 
two visits, but this occurs without benefit of formal intake and several months before 
the youth begins to attend program activities regularly, when did that youth become a 
client? What if the program spends a lot of time (say, up to half a full-time employee 
(FTE) when it only has two paid FTEs) talking to and advising youth who never attend 
regularly-are these youth clients? Is it fair to expect the program to affect their lives, 
as is implied by including them in an outcome evaluation? On the other hand, is it 
fair to exclude these youth from an evaluation, even if there may be more of them in 
raw numbers than youth who attend regularly? How does the program get "credit" for 
them? 

A more difficult issue is what to do with youth and others, such as parents, 
who benefit from a program’s prevention activities without ever going through an 
"intake" procedure. A program may reach many youth and adults through classroom 
or community presentations, without maintaining a list of participants. One option 
for evaluations is simply to count the number of such people reached, or the number 
of presentations made. Another option is to conduct pre-post surveys of these non- 
client participants’ knowledge, attitudes, and behaviors that the program is trying to 
change. Yet a third alternative is to try to assess community-wide impacts by 
surveying the general public for knowledge about the program, perceptions of its 
impact, and perhaps the community’s standing on knowledge, attitudes, and 
behaviors with respect to the prevention topic. 

Defining A Client by Exit or Completion Status 

A standard evaluation approach is to assume that some standard service 
package is "the program," and to begin measuring program impact from the time 
when clients have completed "the program." But for many programs for at-risk youth, 
this approach entails some significant drawbacks. 
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Programs for at-risk youth typically are flexible in the service provision, and do 
not penalize youth who do not come consistently or who do not participate in some 
program components. This program orientation has important implications for 
designing an appropriate evaluation. Irregular program attendance may be simply a 
fact of life for at-risk youth, since many lead relatively chaotic lives (or their parents 
do). Doing anything regularly may be difficult for them. Even if the youth are 
consistent attenders, the program may not have a set of core services, or its "core" 
may include only a small proportion of the service and activity options the program 
makes available to youth. 

One source of this trouble, common to many youth-serving agencies, may be 
that the program tries to operate as a club, membership organization, or family. Once 
attached, users/members are encouraged to stay around for years, perhaps changing 
roles as they grow older (e.g., becoming mentors themselves), perhaps coming around 
less but still dropping by on occasion. There is no set intervention or group of 
services that everyone receives, nor is there a level of performance which, once 
achieved, is considered completion. 

The issue of when a client has left the program is not unique to programs 
operating as clubs. Programs of many other types also have difficulty specifying what 
they consider to be "program completion," and many approaches may be taken. Some 
programs will have a well-defined set of core services or activities which participants 
are expected to complete. Those who do so can be considered program graduates 
(although they still may not leave). Other programs may have a status or role (such 
as counselor or peer mentor) which, if attained, means a youth has graduated from 
the program’s basic activities to a different level. An evaluation might consider such 
youth to be finished with the program. When programs have neither a well-defined 
set of services or a i Parker for graduation, it may be hard to tell who has finished. 

For these programs, one would want to structure follow-up in terms of time since 
program entry rather than in terms of time since finishing the program. 
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At-risk youth may stop coming to a program at some early point because they 
do not feel it meets their needs, or because of problems related to accessibility, or 
because they do not get along with program staff. If an evaluation defines program 
participation at a specified minimum level of involvement then the probability of 
selection biases is increased. That is, if the intervention group is defined as those 
who received the full program intervention, they are likely to differ from those who 
drop out. Any differences observed by the evaluation might then be a function of 
initial group differences (self-selection) rather than a function of the actual 
intervention. 

Handling "Clients By Association" 

Boyfriends /Girlfriends . Problems similar to those faced with the infrequent 
participant arise in considering individuals whose contact with the program is 
peripheral to that of a primary client. In adolescent pregnancy programs, this issue 
frequently arises for males and sometimes also for family members. Some programs 
only address the service needs of males if they are the boyfriends of the girls who are 
the program’s primary clients and they do not consider the males to be clients in their 
own right. Other programs help teenage males whether or not their girlfriends are in 
the program and do count them as clients. These programs may spend equal 
amounts of time helping males, but if the evaluation uses the program’s definition of a 
client, the efforts of the second type of program will "register" in an evaluation while 
those of the first program type will not. 

Parents /Family Members . Parallel problems arise in deciding how an 
evaluation should handle services to families. Many programs try to get parents 
involved, often as adjunct "staff’ or as coaches trying to reinforce the program’s values 
for their own children. Service integration programs may address family needs 
directly because the family’s situation is adversely affecting the youth in the program. 
For example, programs may help parents get drug treatment, or housing, or income 
supports, or job training, or parenting skills training to reduce abusive behavior. In 



such cases, who should be considered the client? If the family is the primary client 
(as is the case in one of the sites we will visit) then the situation is reversed. We must 
ask whether each child should also be considered a separate client, even if a given 
child may not participate in program activities. 

Neighborhood or Community as "Client." Even further frQm the "standard" 
service delivery model is the situation in which a program is trying to change 
conditions in a whole neighborhood. If a program’s target is a whole neighborhood, it 
may not be at all appropriate to use an evaluation design based on the experiences of 
individuals who are in direct contact with the program. Rather, some type of 
neighborhood survey or other aggregated data in which a random sample of 
neighborhood residents respond to questions measuring important outcome;}, may be 
more appropriate. Such a survey could also assess changes (increases) in parental 
and other adult participation in PTAs, tenant councils, chemical dependency 
treatment or prevention programs, and other signs that the community’s adults are 
taking on more neighborhood responsibilities. It is also possible to use observations 
and unobtrusive measures, as Schinke, Orlandi and Cole (1992) did in counting the 
number of crack vials and needles found on streets or the number of shooting 
incidents around the neighborhood in an evaluation of a program designed to reduce 
drug involvement in housing projects (both decreased after the program began 
operating). 

Implications 

There are no right answers to the question of "Who is a client?" but it is a key 
question. If only one program is involved in an evaluation, the answers for that 
evaluation should be negotiated between the evaluators and program staff until both 
are satisfied that the program will be fairly represented by the clients/users included 
in the evaluation. If an evaluation covers a number of programs, even more 
negotiation will be necessary to reach a common definition of program entry and 
program exit that all can agree on and that does not seriously misrepresent some of 



the programs involved. Further, the evaluation design may have to be somewhat 
flexible to accommodate program differences. These agreements may include different 
classes of clients—e.g., youth, families as a whole, boy /girlfriends or siblings who are 
not primary clients themselves, and so on. 

A solution to the problem of identifying program clients is to divide the 
evaluation design into several components. For the fully-participating clients, 
standard and thorough evaluation procedures would be applied. For prevention 
clients such as those reached through classroom outreach, or for "clients by 
association," the evaluation can design an approach that is appropriate to their level 
of involvement and probable program impact. The same can be done for 
neighborhood impact. The critical point is to recognize when designing the evaluation 
that it may not be appropriate to treat all persons in contact with the program 
identically for evaluation purposes, and to adjust the design accordingly. The design 
can be structured to accommodate different approaches for each major way that 
clients come into contact with the program. 

If the definitions finally negotiated do omit some significant numbers of youth 
or other people who have had program contact, the evaluators should develop some 
way to reflect this level of effort even if these people will not be included in formal 
follow-ups and impact assessments. Often simple counts will do, along with an 
assessment of how much time the program commits to this type of contact. For 
example, in the multi-site evaluation of adolescent pregnancy programs funded by the 
Office of Adolescent Pregnancy Programs (Burt et al. 1984), programs reported both 
the number of non-client counseling and referral calls they handled and the number 
of hours they devoted to this effort. Often, these calls were from pregnant teenagers 
who were not sure the program was right for them; program staff spent a good deal of 
time talking with them until they decided, but had "nothing to show for it" if the teen 
decided not to join the program as a new client. With these "non-client" data, 
programs were able to show funders that their support was being used to serve the 
community in ways that complemented service delivery to formal clients. Of course, if 



a significant proportion of program effort goes into these activities, an evaluation that 
focuses on outcomes for the more intensive program services may not actually assess 
significant aspects of the program’s impact. 

A further implication of the foregoing is that the package of services offered by 
youth-serving programs is usually too multi-faceted and too flexible for an evaluation 
to use "program exit" or "program completion" as the point at which impact evaluation 
begins. For these programs, it seems much more appropriate to use the point of 
program entry as the time to begin. This decision, of course, has its own implications 
for the thoroughness with which service delivery must be measured. 

Differences in Client Risk Levels That May 
Affect Services Received and Evaluation Plans 

Programs for youth may serve a very wide age range (10-19, and sometimes 
even older). Youth of different ages within this range are likely to have very different 
needs, and to experience very different risk probabilities. Youth programs are quite 
likely to offer some combination of preventive activities and treatment services, and 
may also facilitate access to housing, income maintenance, and other concrete 
services for the families of youth in the program. Because of the age range of interest, 
it is likely that programs will serve some youth whose situations are only moderately 
risky alongside others who are already in serious trouble. As a result, activities and 
services offered to youth are likely to differ widely. Some programs may attempt to 
serve all youth; others will specialize in a particular age group or in youth engaging in 
a particular type of problem behavior. Cross-program evaluations and evaluations of 
programs serving a wide range of ages and risk levels need to decide how to 
incorporate and understand the effects of this diversity. 



Effects of Risk Level on Selection into a Program 

A client’s risk level may affect which program a client enters. Youth at low risk 
may enter youth development or prevention programs where activities are the primary 




67 



n 



t 



focus, whereas youth at higher risk (or youth who are already in trouble) may enter Or 
be placed in case management or treatment programs. 

The biases involved in the effects of risk on selection into a program must be 
faced by any multi-program evaluation. Such evaluations need to be sure that the 
programs included in the evaluation are all serving youth with roughly similar risk 
levels, or else that the design includes enough programs serving youth at different risk 
levels so that researchers can analyze differences within and between programs 
grouped by the average risk level of their clients. 



Effects of Risk Level on Mix of Services Received 

A client’s risk level may affect which service components of a comprehensive 
program he or she is offered. The biases involved in the effects of risk on service 
delivery within programs are faced by every evaluation. The inconsistencies of 
program delivery (which the program sees as flexibility) offer a number of 
opportunities as well as challenges for evaluators. An evaluator can examine the 
process by which programs determine who needs what--programs may appropriately 
offer a different mix of services to different youth. An important evaluation question 
is "How do programs determine who needs what?" 

In programs that emphasize activities over services, as many youth-serving 
programs do, some service needs may be overlooked. In any program, a youth is only 
likely to be referred for services if a staff person has become aware that the youth has 
some service need. In heavily activity-oriented programs, routine and comprehensive 
needs assessments may not be done. For instance, in mentoring programs, a youth’s 
mentor may not make a systematic effort to identify new service needs as they arise, 
even if a program staff person has conducted an initial needs assessment. Therefore 
service delivery may look erratic in these programs. An important evaluation issue in 
the context of programs set up this way is whether the program misses many existing 
service needs. It might also be important to address what happens to youth when 
their needs are not met. Either of these evaluation options requires the evaluator to 
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conduct needs assessments for all clients. Such evaluation activities may be seen as 
disruptive to the program, but they have been negotiated in some instances. In order 
for this to happen, the program or the program funder must care about whether the 
program identifies and addresses most of its clients’ needs. 

No evaluation should make the assumption that an organization delivers "a 
program" similarly for all clients, regardless of risk level. Therefore every evaluation, 
including those for youth-serving programs, needs to plan to collect measures of 
initial client risk status (also sometimes called client difficulty). These will be used in 
outcome analyses to qualify any observed results, either by analyzing results 
separately for different risk groups, or by entering initial risk level as a covariate or 
control variable in regression, ANOVA, or other statistical treatments. The former is a 
safer approach since the latter assumes that risks have been measured accurately. 

Client risk levels may be used in analysis to understand or qualify evaluation 
results. Study participants who show improvements may have a relatively low risk 
level. The program may only have helped those with some pre-existing competencies 
and skills. In other cases, the program may have been most helpful to those who 
were the least functional when they entered the program. For example. Project 
Redirection used initial risk level information (in school or dropout; AFDC recipient or 
not) and participation levels (months of active program participation) to understand 
which teenagers received most benefit from the program. The analysis indicated that 
those teens who benefitted most were those who faced the greatest obstacles to self- 
sufficiency at program entiy (Polit, Quint, and Riccio 1988). In either case, 
recognizing the impact of client risk levels and planning the evaluation so they are 
available for use in regression or other multi-variate analysis will increase the 
accuracy of interpretation. In some cases using client risk levels can prevent 
researchers from drawing false conclusions, as would have happened if the Project 
Redirection researchers had stopped their analysis when results showed no effects for 
all clients taken together. The real impact of the program was only visible when 
clients were grouped by risk level. 



Implications 

It is crucial that multi-program evaluations plan for the high probability that 
client risk levels will differ between programs. It is almost as likely that single- 
program evaluations will encounter clients with very different risk levels. Evaluators 
must develop designs that can assess the effects of varying risk levels on outcomes of 
interest. This means that programs being evaluated must have record-keeping 
procedures (and preferably a management information system) capable of recording 
both the problems of youth at risk who are actual clients and the types and amount of 
services each youth receives (Jacobs 1988). Since the quantification of risk is a new 
and highly experimental enterprise, where acceptable levels of reliability and validity 
have not been adequately demonstrated (Wells, Fluke, Downing and Brown 1989), 
recording risk levels will not be simple. We would approach this by having programs 
record at intake the presence in a youth’s background of factors (antecedents, 
markers, problem behaviors, risk outcomes) included in the risk model described in 
Chapter 2 and in more detail in Resnick et al. (1992). Then the evaluation will have 
the information and can use the variables as controls in any combination where 
needed in the analysis. Further, using multiple measures of risk to create a 
composite score reduces measurement error and yields better results. 

To conduct analyses such as those just described with sufficient statistical 
power to detect subgroup differences in outcomes, an evaluation must have planned 
for a large enough sample to create subgroups of adequate size. For many of these 
programs, it may take longer than expected to assemble the required sample and sub- 
sample sizes. Longer evaluations may be more costly, and will certainly take longer to 
produce results. 
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Documentaticn/What's the Program? 

It is our belief that no agency actually delivers "the program on paper" to each 
one of its clients. Even the most carefully structured and precisely defined program 
will not be able to treat every client exactly the same. Most programs do not attempt 
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such uniformity, and some consider it contrary to their philosophy. Even curriculum- 
based interventions vary from teacher to teacher, although all children in a single 
classroom presumably are exposed to the same input (if they are not absent, and if 
they are paying attention). The best approach to documenting each possible program 
configuration is to be sure the evaluation obtains data on actual service delivery, 
including participation in activities, for each client and each activity or service. A 
management information system will facilitate this type of data collection. At the very 
least, a manual method of recording client participation or service receipt must be in 
place and must be used. 

Even case management programs, which specialize in service delivery, may find 
it difficult to record all client contacts and services received. It is even more difficult 
to get programs whose major focus is growth-enhancing activities or recreation to 
record participation or services received. Their emphasis is on keeping the youth 
involved, not on solving a particular problem in a relatively short time period. The 
problem for evaluation is especially challenging if services and activities are handled 
by different people (e.g., a mentor does enjoyable activities with youth, but sends the 
youth to a case manager if specific services are needed). Participation in activities 
can be handled with a daily sign-in log or similar mechanism. It will not get precise 
levels of participation, but the program will probably be happier than if a more precise 
mechanism is required, and data recording the number of days a month the youth 
showed up at the center can serve as a proxy for detailed participation records. If the 
program has some staff who handle service assessments and referrals, these staff 
should record the actual services delivered to youth. 



EVALUATION ISSUES SPECIFIC TO 
SERVICE INTEGRATION EFFORTS 



Documentation of Service Delivery 

A preliminary evaluability assessment is an essential beginning point for any 
evaluation of comprehensive SI efforts. During the evaluability stage, researchers 
identify the services available in the network, the existence and nature of the links 
between program components, and the program’s expectations (hypotheses) for how 
these components will affect client outcomes. This set of clear predictions lets the 
evaluation distinguish between intended and unintended program benefits. 

Specifying the exact services which comprise program "components" also lets the 
evaluator track the operation and implementation of these components during the 
formative evaluation stage. 

The ability to link individual service components to individual clients and their 
specific program outcomes is not part of "black box" program evaluations. A "black 
box" evaluation is one which assumes that the treatment group gets "the program," 
that the control group does not get "the program," and that the evaluator knows what 
"the program" is without having to measure actual program delivery. In reality, it is 
the very rare program that is delivered virtually identically to every participant. 

It is relatively common for programs to refer their clients to other agencies for 
needed services without having any system in place to get feedback from the referral 
agencies as to the client’s actual receipt of services. In these programs there is no one 
file that contains all the information about a given client’s receipt of services. 

Programs may even resist the need to know whether clients got the services for which 
they were referred. They see their responsibility as making the referral; and the 
client’s responsibility to follow through. It is critical that the evaluators of SI projects 
develop a mechanism for obtaining feedback from referral agencies about the actual 
delivery of services. 
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We assume it is essential that SI efforts maintain accurate records of service 
delivery, to do justice to a program offering comprehensive services (where most 
clients will not get many services, but any client can get a service if needed). During 
the evaluability assessment stage, researchers should examine the program’s current 
practice and future ability to record service delivery on a client-by-client basis. To 
provide maximum flexibility in analysis and adequately represent the program as 
delivered to clients, it is important to have, or to develop, a systematic method for 
recording who got what services and who participated in what activities. If the 
program also relies on inter-agency collaboration to supply some or many program 
services (SI), documentation of service delivery on a client- by-client basis should be a 
core component of any evaluation for formative purposes as well as for outcome 
analysis. 

SI Network Relations and Structure 

All service agencies that are part of the SI network should share roughly the 
same ideas about what services are being offered and how these services fit into the 
overall design of program inputs and outcomes. However, in an SI effort involving 
many and diverse agencies, it may be difficult to develop this common understanding. 
An evaluability assessment may reveal important differences of opinion among the 
coordinating agencies and these may have short-term negative effects on service 
delivery and planning. The evaluator who works with the program to develop its 
evaluation plan must anticipate these problems, and be sensitive to any unintended 
consequences of the evaluation itself on the cooperating services. 

Further, SI may work in any of a variety of formats. If the program design is 
one in which youth enter through any of several co-equal agencies in a network, each 
of which retains "their" youth as primary clients and provides case management 
services, there could be as many images of "the program" as there are agencies in the 
network. Since each may have a somewhat different emphasis, youth attached to one 
program may receive a very different set of services from that received by youth 



attached to another agency. Another model, more common than the one just 
described, is a central youth-serving agency which provides an array of activities and 
services itself and also establishes inter-agency linkages for the services or 
entitlements it cannot offer or needs only rarely. Both of these models could be 
evaluated for the effects of SI on the ease, frequency, volume, speed, and other 
aspects of service delivery. But for the first model it might not make much sense to 
ask about the effects of "the program" on all of the youth served by agencies in the 
network. 

Non-client Outcomes of Interest 

Documentation of comprehensive service integration programs should include 
an assessment of the effectiveness or efficiency of the referral network. Many of these 
programs rely heavily on informal inter -institutional linkages with existing service 
agencies; other linkages are formal and explicit. The literature on inter-agency 
cooperation discusses the nature of social agency "service boundaries" and their 
"permeability" or "rigidity." Of course, agency rigidity may be merely a reflection of the 
rigidity of their funding sources or the benefit programs they administer. Overly rigid 
agencies or benefit programs maintain many restrictions on client eligibility, and these 
restrictions have been associated with clients not receiving services from the referral 
agency, despite making contact. The Office of Technology Assessment (1991) 
identifies this factor as a major impediment to traditional service delivery for at-risk 
youth; it is also a prime reason for attempting SI, whose purpose is to increase 
permeability. 

The ability of agencies in an SI network to work out more flexible and 
"permeable" boundaries will certainly affect service delivery and will probably also 
affect client outcomes. Eval jations of SI programs should docun nt how the 
networked agencies developed more flexible procedures (if they did) and describe the 
changes in agency flexibility that resulted. Gomby and Larson (1992) suggest a 



variety of indicators which can be used to document the system and its service 
delivery effects: 



System effects: 

• Memoranda of understanding between agencies (should be some/more): 

• Waivers to use funding streams in innovative ways; 

• Steering committee with multi-agency representation (should be one); 

• Frequency of meetings among participating agencies (should increase): 

Service delivery effects: 

• New, simpler forms; 

• Number of contacts clients have with multiple agencies (should go up); 

• Time spent waiting for services (should go down); 

• Referral patterns (should become more creative and appropriate); 

• Services delivered to one agency’s clients by other agencies (should go up, but 
also should be more appropriate); 

• Services used by participants (should increase, and also should be appropriate 
to participants’ needs); 

• Services offered by participating agencies (agencies might fill in gaps in service 
system, or might alter their service mix to avoid duplication). 



To these we might add the following system effects: 

• Increased personal contacts and comfort of agency staff across service systems 
(e.g., among education, juvenile justice, mental health, income maintenance); 

• Increased knowledge among case managers of services available; 

• Complete inventory and reference book of services available in the community, 
their eligibility criteria, and how to apply; 

• Extent to which agencies use collocation of staff, staff exchange programs, 
multi-agency teaming. 



The first two additional indicators of system effects would require questionnaire or 
survey assessment. The last two additional indicators may be documented from 
existing records (e.g., the reference book will exist; the staffing patterns will be 
documented through memos and agreements). 

An SI issue that may affect program replicability is the variability in community 
service networks. The experts we interviewed for this project reported that youth- 
serving programs develop their specific service configurations in idiosyncratic ways, 
often beginning with informal relationships among agency directors (Correia, 
interview; Jones, interview). This means that an attempt to repeat a successful SI 
effort in other communities may not succeed in assembling an array of services 



similar to those of the model being replicated; further, there could as easily be more 
services available in the replication community as fewer or different services. 

It may be important to document what is missing from the service integration 
package in any given program, either because it is completely unavailable in the 
community or because the core program could not, or has not yet, developed a 
relationship with the appropriate agencies. For example, at-risk youth probably 
would not benefit from an employment-oriented peer support program unless jobs 
appropriate to the youth were available in the community. This issue was confronted 
by Halpern and Larner (1986, cited in Halpern 1986) in the Child Survival/Fair Start 
initiative, in which the effectiveness of a program for migrant workers was adversely 
affected by the lack of medical resources to treat conditions once identified by the 
program. 

Differentiating the Impacts of SI 
From those of Comprehensiveness 

Comprehensiveness and service integration are not the same thing. A program 
can be comprehensive by providing all needed services itself, without relying on any 
inter-agency collaboration. A program can be integrated (i.e., use collaborative 
arrangements) and not comprehensive. A program can rely on SI as its mechanism to 
become comprehensive. A program's definition of "comprehensive" may differ from the 
evaluator’s or funder’s definition. We think it is important for any evaluation in this 
area to try to sort out the effects on clients of comprehensiveness from those of service 
integration, as well as the effects of SI on the comprehensiveness experienced by 
clients. 

The most likely service integration impacts that affect clients are improvements 
in the ease, frequency, volume, speed, and accessibility of services not available 
through the core program. It may also happen that the client gets one or more 
services that he or she would not have gotten at all without SI. Then we would want 
to assess the impact of faster, easier service receipt, and also the impact of a different, 
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enhanced service mix. It could be that speed and ease mean that the situation the 
client faces does not have a chance to deteriorate beyond hope. Or, because the 
program can "deliver" when needed, the client keeps coming to the program and 
participating in enhancement activities. It should also be relatively easy to tell if SI 
increased comprehensiveness (assuming that service delivery is recorded accurately 
and fully). However, if the services would not have been available without SI. then the 
effects of SI and comprehensiveness will be confounded, and the evaluation will not be 
able to say anything separately about these two aspects of program configuration. 

Evaluators and program staff should discuss and develop realistic expectations 
of the specific nature of increased comprehensiveness they anticipate from SI, as well 
as of the effects they expect independent of enhanced con prehensiveness . Once these 
effects are identified they can develop mutually acceptable ways to measure these 
effects, including observational or qualitative approaches. They may decide that it is 
too difficult to separate out the effects of SI on comprehensiveness and SI independent 
of comprehensiveness unless one is doing a multi-site evaluation of programs that 
vary systematically in their degrees of SI and of comprehensiveness. 



LESSONS LEARNED FROM PREVIOUS EVALUATIONS 

Substantive Results from Evaluations 
of Youth-Serving Programs 

Resnick et al. (1992) summarized Dryfoos’ (1990) and the Office of Technology 
Assessment's (1991) analysis of important common elements in successful youth- 
serving programs addressing school dropout, teenage pregnancy and parenting, 
substance abuse, and delinquency. Here we briefly report the common elements; the 
reader who desires a more extensive discussion of what makes for successful 
programs should refer to Resnick et al. (1992). Programs in each of these substantive 
areas have certain characteristics that are specific to their problem focus, yet Dryfoos’ 
(1990) review of many evaluations notes that a surprising number of program 
elements recur in evaluation after evaluation regardless of program focus. Successful 
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youth-serving programs--those which evaluations have shown to make a difference for 
youth— are those which: 

• Identify at-risk youth early and intervene early; 

• Provide long-term and consistent intervention, with age-appropriate content 
changing over the years; 

• Provide individualized attention and instruction, including intensive counseling 
as needed; 

• Make comprehensive services available to youth, as needed, through on-site 
provision, co-location, or case management support; 

• Include an emphasis on growth, skills enhancement, life options, vocational 
orientation; 

• Develop and use multiple channels of influence, including community-wide 
support and effort (e.g., media, church, parents/families, neighborhood 
prevention campaigns); 

• Provide a safe and stable physical environment for the program. 

Obviously programs serving youth should heed these findings, and evaluators should 
be sure to include therein evaluation designs. 



Substantive Results from Evaluations 
of Service Integration Efforts 

In his extensive review of studies evaluating twenty years of service integration 
(SI) efforts, Kusserow (1991) extracts several common findings. These evaluations 
reveal that SI efforts have indeed made services more accessible to clients and more 
responsive to their needs. Specifically, these efforts have enabled clients to obtain 
and benefit from services that they otherwise would not have received. Both the 
specific benefits and the general level of impact of SI depended on the commitment 
and communication levels of agency staff in all linkage agencies, and on institutional 
support and agreement from the participating agencies. 

Kusserow concludes that SI efforts have not been sustainable over the long 
run. In particular, he notes that these efforts have not succeeded in institutionalizing 
system change. He lists six barriers commonly encountered by SI efforts that limit 
the degree of system change that can be achieved: 




Size and complexity of the human services system; 
Professionalization, specialization, and bureaucratization; 
Limited influence of integrators; 

Weak constituency for service integration; 

Funding limitations; 

Insufficient knowledge. 
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The SI efforts aimed at programs for at-risk youth also face these barriers. 

Conducting Evaluations 

Who Should Conduct Them? 

The experience of many evaluation efforts suggests that program staff should 
not be expected, on top of their regular duties, to conduct the evaluation or collect 
significant amounts of data for it. They do not have time and they will always place a 
higher priority on responding to the needs of clients than on systematic data 
collection, and this is appropriate to this role. 

This means that the evaluators (those who do the actual work of evaluation) 
should be outsiders. But outsiders may not really understand the program, or be 
responsive to its needs and concerns. For the evaluation to be a good one, the outside 
evaluator needs to take the time to get to know the program and work carefully with 
the program staff to develop mutually agreeable arrangements. As Quinn (1992) 
points out. evaluation funders need to allow enough resources to provide researchers 
to get to know the program. The effect will be to have an "insider’s" outside 
evaluation, which is likely to be more valuable to insiders and outsiders alike. The 
resulting evaluation design and products will be well worth the effort in terms of 
program good will and in terms of a qualitative and quantitative documentation of 
program activities and impacts. 

Working with Programs 

From the program perspective, the best evaluations are those which do not 
disrupt program activities, do not place an heavy burden on program staff, and reflect 
the program and its goals in a positive light. While evaluators may place major 
emphasis on numbers and types of services (e.g.. to demonstrate comprehensiveness, 
or to show the effects of SI), programs may feel this emphasis does not reflect their 
overarching purpose of youth development, growth enhancement, or leadership 
training (Pittman and Cahill 1992). As discussed later ("Identifying Realistic 
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Outcomes and Impacts to Measure"), those impacts which programs care about most 
may be the most difficult to measure adequately. 

Since most youth-serving programs do not evaluate themselves, the claims they 
make to their communities about program impact may be exaggerated. Such 
programs may fear evaluation, because they fear the data will not support their 
claims. It is critical for evaluators tc work with programs until the programs 
understand the possible benefits of evaluation and are prepared to support the 
evaluation effort. 

Working with programs until they are happy with evaluation plans is relatively 



easy when only one program is involved and that program has hired the evaluator. 

For evaluations imposed from outside, the situation is sometimes more difficult. It 
reaches maximum difficulty in multi-site evaluations. 

Multi-site evaluations usually occur when a foundation or Federal funder 
provides financial support and cooperation with the evaluation is a condition of 
receiving project money. Often the funder, not the program, specifies the goals and 
outcomes to be examined. This situation needs to be handled very delicately to avoid 
alienating the programs involved, since each program may have a different service 
configuration and interpret success in its own way. Occasionally a youth-serving 
organization with many affiliates will undertake its own evaluation, as did the Boys 
and Girls Clubs of America (Schinke, Orlandi, and Cole 1992), Girls Inc. (Smith and 
Kennedy 1991), and Big Brothers/Big Sisters of America (cited in Quinn 1992). The 
Center for Substance Abuse Prevention (DHHS) funded the first two evaluations and 
foundations funded the third through the evaluator Public /Private Ventures. In these 
three cases, the parent national organization controlled the evaluation and took pains 
to structure the work to be compatible with national and local goals and with the 
workload and operating procedures of local affiliates. 

For an evaluator, there is a significant difference between a situation where a 
program funder has required an evaluation and one where an evaluation is requested 
by a parent organization. In the former situation the evaluator ultimately answers to 
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the program funder, while in the latter the evaluator answers to the program. 
Evaluators involved in the first type of evaluation should try to shape their behavior 
as if they were involved in the second type, if they want to gain the greatest degree of 
cooperation from the programs. This may take some diplomatic negotiating, to 
simultaneously remain responsive to the funders questions. 

Quinn (1992) details many suggestions for maximizing the mutual satisfaction 
of programs and evaluators. These include: 

• Include planning for evaluation as an integral part of planning for the program 
itself: 

• Help program staff understand in non-technical ways the different types of 
evaluation, their purposes, and what they can do for programs; 

• Involve staff in decisions about what level of evaluation to conduct, in 
specifying important program outcomes, and in defining measures of success 
that make sense to the program: 

• Recognize and work with the "daily life" of a program, including potential 
difficulties with random or quasi-experimental design, the flow and flexibility of 
program activities and youth participation and plan the evaluation accordingly: 

• Include in the evaluation design plans to document the community context, 
service system context, and other contexts in which the program operates, so 
program staff see that the evaluators understand the program and will be able 
to present it accurately to the outside world. 



Evaluators who follow this advice will produce evaluations that are more useful for 
both the program and the sponsor. 



Additional Methodological Issues 

Much of the discussion so far has addressed specific methodological issues in 
planning and conducting evaluations. Here we note the importance of qualitative as 
well as quantitative data for understanding programs, and reiterate the need for 
detailed service use data. 

Evaluation planners should consider augmenting quantitative records of service 
delivery with qualitative and observational methods, including ethnographic methods. 
While these methods often are decried as lacking validity, they actually enhance the 
validity of interpretations of more quantitative results. This situation arose in the 
Child and Family Resource Program (CFRP) evaluation, an early childhood 
intervention to prevent school failure (Travers, Nauta, and Irwin 1981). The 



quantitative data showed no improvements in the children’s cognitive abilities, which 
suggested that the cognitive stimulation curricula, delivered via home visits, was not 
effective. However, the ethnographic component to this evaluation found that the 
home visitors often were not able to deliver the planned curriculum because they had 
to help the parents deal with more concrete living problems such as housing evictions, 
physical safety, and financial problems. The CFRP did not serve 10- to 15-year-olds, 
but the experience of its evaluation has important implications for evaluating 
programs targeting high-risk youth. 

As discussed above, the standard "black box" approach to evaluation and an 
exclusive focus on outcomes/impacts has its limits (Cronbach and Associates 1980). 
Generally, outcome evaluations from related fields such as the early childhood arena 
are instructive because they typically show what evaluations have not achieved. 
Outcome evaluations have not been able to identify which program components are 
most effective under what kinds of local conditions. Rather than asking the question 
of whether a program "works," we should be asking how it works, what components 
are the "active ingredients," under what conditions, and for whom (Weiss 1983). Not 
knowing these specifics about program-client fit makes it harder to recommend future 
applications of a demonstration program or to translate results into broader policy 
directions. Our earlier discussion of service documentation spoke to these issues and 
what evaluation planners can do to address them. 

Maintaining Levels of Service guality 

Program quality may be affected by a host of factors, including the initial 
funding level, the source of funding, the commitment of participating service agencies, 
staffing, and changes in policies or legislation that affect service provision. For 
innovative community -based preventive programs, of which comprehensive service 
integration programs for at-risk youth comprise one type, one of the most important 
influences on the quality of the program is reliance on large-scale research and 
demonstration (R&D) projects to develop the design, oversee the implementation, and 



monitor the effects of new programs. These R&D projects are typically funded by a 
foundation, such as the Annie B. Casey Foundation’s New Futures program, or a 
public-private partnership, such as Public/Private Ventures Inc. of Philadelphia, or by 
Federal program initiatives. 

Certain features of R&D efforts make them attractive as ways of "proving" the 
effectiveness of innovative program models. They tend to be funded at levels that 
provide optimal program quality. They usually have an initial planning stage 
involving consultations from a variety of experts on how to maximize program effects. 

A rigorous evaluation plan is designed as part of the overall project (and indeed, the 
evaluation is mandated as a critical piece of R&D). The evaluation is carried out by 
independent, professional researchers. Finally, since the evaluation is part of the 
project from the beginning, program personnel are more likely to accept and support 
it. All of these factors help assure that the best possible "program model" is 
implemented and that any program outcomes will be identified through careful and 
rigorous evaluation methodologies. 

However, standard R&D evaluations usually stop one stage too soon, and their 
results therefore often have little relevance for "real world" applications. The problem 
with R&D efforts occurs at the end of the program development cycle, when the 
evaluation has demonstrated the program’s benefits. At this point, policymakers who 
support the program often find that they have inadequate resources for program 
expansion or they attempt to expand the program without providing additional 
resources or support. Usually, public funders cannot afford to replicate an R&D 
model as it was demonstrated and disseminate it widely at the same time (Weiss 
1988). 

These constraints may lead to "replication" programs that are pale imitations of 
the initially successful research and demonstration project. One example is the case 
of tile Prenatal Infant Development Project (Schorr and Schorr 1988), and there are 
others, particularly for early education programs based on the Perry Preschool Project. 
Heather Weiss has labeled this process the "demonstration-dilution effect" (Weiss 



1988). An ideal evaluation design would extend its examination of program impacts 
into the replication stage and ask the critical but often ignored questions— "What is 
the minimum amount of this program that can be expected to have an impact?" and 
'What are the essential elements of this program which, if diluted or eliminated, 
materially change what one can expect the program to accomplish?" 



EVALUATION ISSUES SPECIFIC TO THE 
TYPE OF SITES SELECTED FOR THIS STUDY 



Is the Program Ready for Evaluation? 

Most programs, whatever their stage of development, can benefit from formative 
evaluation efforts. But not all programs are ready for summative or impact 
evaluation, and it is summative evaluation that we focus on here. There is some real 
question as to whether comprehensive SI programs for at-risk youth are ready. Those 
that have been in stable operation for ten or more years are certainly ready. Others 
that are just beginning to assemble their network and negotiate inter-agency 
agreements are probably not ready. It has been said that the major weakness of past 
summative evaluations has been the premature use of experimentation (Mark and 
Cook 1984). In order to conduct an effective (meaning, valid) summative evaluation, 
extensive prior knowledge of program operation is required, including knowledge that 
the treatment is well developed, that it will be implemented as planned, that measures 
are available, appropriate, and well-developed, and that the "ecology" of the program is 
well understood (Mark and Cook 1984),. As Cronbach and Associates (1988) state, "do 
not evaluate until you can do the program proud." 

Typically, one begins any potential evaluation endeavor with an evaluability 
assessment (Schmidt et al. 1975; Wholey and Newcomer 1989). During the 
evaluability assessment process, researchers identify the expected short, medium, and 
long-term goals of the program, the program components that are designed to produce 
the desired outcomes, and the assumptions underlying the connection between 
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program inputs and outcomes, and whether outcomes are measurable (Ruttman 
1984). Understanding the community context in which the program operates is also 
important. 

Certain circumstances might render a full-scale summative evaluation of a 
program unreasonable. The program might be very new, undergoing major 
reorganization, or experiencing widespread staffing changes. The program might not 
have the ability, or the willingness, to record the types and amounts of data that a 
full-scale evaluation would require. The program might be stable and have a good 
record-keeping system, but lack a clearly articulated set of goals for clients and/ or 
reasons for offering the services they do. Or, they may have goals for which no 
reasonable measures exist. Finally, the program as it actually operates might not fit 
the evaluator’s interests— in our case, the service linkages might be so fragile, casual, 
or opportunistic that we would not consider the program to be engaged in an SI effort. 

Assuming that a comprehensive SI program for at-risk youth is evaluable and 
that the effects of SI are at least theoretically separable from those of service delivery 
per se, there are three major evaluation design issues that we will discuss here: what 
outcomes to measure, what to compare the results to, and how to reduce attrition to 
follow-up. The first addresses the question of what happened to program participants 
and how we can measure it. The second addresses the question of whether the 
program was the cause of the observed outcomes or whether they might have 
happened even without the program. The third addresses the issue of whether the 
results will be biased because the evaluation could not obtain follow-up data from a 
significant number of program participants, who may differ in some systematic way 
from the people whom the evaluation was able to reach. 

Identifying Realistic Outcomes and Impacts to Measure 

A program’s stated goals are usually the starting point for identifying 
appropriate outcomes for an evaluation to measure. Once these goals are identified, 



the evaluator can determine the best measures available and feasible in the particular 
evaluation setting. 

Measuring the attainment of some goals is easy. If a program tries to keep 
youth in school until the end of each school year, this is an easily observable 
outcome. If the program tries to assure that teenagers bear healthy babies, the birth 
of a full-term normal weight baby with normal APGAR scores is a clear measure of 
success. Both of these outcomes affect a public system; the school or the hospital 
records the outcome, and the program simply has to access the system records. 

However, efforts to use existing secondary source data such as school records, 
reports from participating service agencies, or arrest records as indicators of program 
impacts may be complicated by privacy regulations. Confidentiality agreements 
within and between agencies that restrict evaluator access may limit the utility of 
these data sources for evaluation purposes. How the lead agency defines its 
agreements with the other member agencies concerning shared information may affect 
the availability of secondary data sources. 

Other goals have clear outcomes conceptually, but these outcomes may not 
generate system markers. For instance, programs trying to prevent substance use or 
criminal behavior have a conceptually clear outcome--the youth either engages in the 
behavior or does not. But many youth may engage in the behavior without getting 
caught, and some youth who use drugs or commit crimes relatively infrequently may 
get caught on the rare occasions when they do so. Public systems are poor recorders 
of these outcomes. In addition, youth may or may not be willing to tell an evaluator 
what they have been doing or may not be able to recount their activities accurately. If 
we could observe the youth at all times, we would know whether program outcomes • 
were achieved. But no evaluation will ever reach this level of surveillance. For some 
outcomes one can make random observations (e.g., random urine tests for drug use), 
but most youth-serving programs probably would consider such observations 
unacceptably intrusive and disruptive of the program and its relationship with clients. 



Further, since one can only randomly observe those clients one can contact, the 
issues of follow-up and attrition discussed earlier are pertinent. 

Measuring Individual Outcomes 

Programs may try to affect their clients’ knowledge, attitudes, and/or behavior 
with respect to a wide range of topics. Prevention programs usually target particular 
behaviors associated with risk (not using drugs, not smoking, abstinence from sexual 
activity). They may try to change the behavior directly, but they will often try to 
change knowledge about the risks associated with the behavior and attitudes toward 
the behavior as a means to affect behavior change. Knowledge and attitudes are 
typically measured with paper -and-pencil instruments; often these are administered 
immediately after an intervention, and the results compared with responses before the 
intervention. The important extension of this methodology to assessment in some 
follow-up period is less often done, but much research indicates that the effects of 
short interventions aimed at knowledge and attitudes often wear off relatively quickly. 

Changed behavior is an important thing to measure, whether the program is 
primarily a prevention program or a treatment program. The nature of the behaviors 
may be more complex in treatment programs (e.g., counseling with a youth and 
parents may attempt to change long -ingrained habits of interaction and 
communication), but measuring the presence or absence, increase or decrease of 
behaviors is usually an essential element of program evaluations. Self-reports 
(through interviews/questionnaires) and system markers (school, agency, court 

t 

records) of behavior are common evaluation tools. 

Some program objectives may not be clear conceptually, and therefore will be 
difficult to measure. Many programs try to increase youth self-esteem, or promote 
growth or leadership ability. Some of these goals do not have readily available, 
standardized measures with sound psychometric properties, as is the case, for 
example with measures of the quality of parent-adolescent interaction (Howrigan 
1988). Although measures for many psychological characteristics exist, many were 
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developed strictly for an academic study sample. There are typically no norms for 
these measures, so one would not be able to say that the youth of a particular 
program score as high or higher than, say, 70 percent of youth in the nation. If one 
did use scales with national norms, the norms would provide a natural "comparison" 
group. Other measures assess only attitudes and perceptions, not the actual behavior 
of interest. In addition, measures developed in laboratory settings that do assess 
behavior may not be appropriate to a program evaluation because the measurement 
activities take too much time and money (often involving one-way mirrors, videotape 
recordings, and trained coders). 

What usually happens is that evaluations fall back on measures that Eire 
available or feasible rather than measures that are meaningful in evaluating these 
programs. This is a documented shortcoming of many previous evaluations of 
innovative programs for children and youth, dating back to the early evaluations of 
Head Start. Despite the mandate of Head Start to influence a broad range of 
outcomes including children’s health status and parents’ community involvement, 
more than half of Head Start effectiveness studies focused primarily on children’s IQ 
scores (Hauser-Cram and Shonkoff 1988). 

Measuring Community Impacts 

Comprehensive service integration programs for at-risk youth also feature a 
wide range of potential program goals, including changes to the participants, the 
families of participants, participant peer groups, and changes in inter-agency linkages 
and the larger community environment. While measures exist for youth skills 
(particularly school performance), social skills, and "problem behaviors," proven 
assessment instruments become scarcer as one moves farther away from youth and 
their concrete behaviors as the focus of measurement. A "Catch-22" situation may 
arise, where programs use narrow outcome measures to assess complex, ecologically- 
oriented programs, simply because there are so few valid and reliable alternatives 
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(Weiss 1988). But the measures chosen do not reflect most of the effort of the 
program, or its intended effects. 

If the program is designed to change a whole community, then the appropriate 
measurement will be at the community level. Schinke, Orlandi, and Cole (1992) offer 
an excellent example of measuring community impact; in their case of the effect of 
Boys and Girls Clubs on substance abuse, parental involvement, and general 
neighborhood disorganization in public housing projects. Substance abuse was 
measured by discarded containers and drug paraphernalia found on project grounds; 
parental involvement was measured by participation in tenants associations, youth 
organizations, and schools; and general neighborhood disorganization was measured 
by vandalism and graffiti in unoccupied housing units. Parental involvement 
increased and the other indicators decreased in projects that had Clubs; projects with 
Clubs also fared better on each measure than comparison projects without clubs. The 
results led to widespread entree into Public Housing Authorities for the Boys and Girls 
Clubs, which now operate over 100 clubs on housing project grounds. 

Measuring Change 

Another issue in the selection of appropriate outcome measures is the degree to 
which the instrument can detect change over time (and whether the instrument is 
sensitive to relatively small program effects). Many scales and other instruments to 
measure outcomes are derived from laboratory studies of child development which 
attempt to assess a child’s competencies at one point in time. Not all such measures 
are good at reflecting change. This limitation of the existing measures may be due to 
the nature of the test items (especially those measuring risky environments, which 
may not change over time), the lack of test sensitivity, or the inappropriateness of the 
measure as a operational test of a given program goal. 
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Measuring Behaviors Prevented 

Trying to measure things that did not happen (successful prevention) poses 
additional difficulties. Sometimes evaluators try to measure prevention indirectly, by 
assessing increases in skills or competencies which may inoculate the recipient 
against the risk conditions (Bloom 1979). Sometimes an evaluation uses a population 
rate of something that registers as a marker in some social agency’s data system (e.g., 
city- wide drug arrests for youth; city-wide teen birth rates), even if the intervention 
has been addressed only to a very small proportion of the whole population (e.g., one 
neighborhood). For example, primary prevention programs might be assessed by 
measuring city-wide or county-wide arrests of juveniles, or teen birth rates, or total 
school dropout rates, but the program only operated in one community representing 
only 5 percent of the youth in the city. This practice is clearly an unfair measure of 
program impact, since it is very unlikely that the program could have affected a whole 
community. If at all possible, the evaluation should seek system data at the 
neighborhood level, to assess impact on the population actually reached by the 
program. 



Issues for Multi-Site Evaluations 

Multi-site evaluations pose their own problems for selecting outcomes to 
measure. Outside evaluators may impose common outcome categories on all 
programs in a multi-site evaluation. But this common set of outcomes may not reflect 
significant aspects of each program in the evaluation. To some extent, the selection of 
outcome measures might need to be specific to each program site because at least 
some of the program goals will be highly site-specific. 

Even programs with the same nominal goals may prefer different indicators of 
goal attainment. These differences may be a function of the way in which the program 
was implemented at the given site, or they may stem from differences in decision- 
making processes at different program sites. For example, some sites might not allow 
evaluators to use some measures which they consider overly intrusive, but other sites 
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may have no problem with the measures. The result may be a great deal of inter-site 
variability in which program goals get included in the evaluation, whether major 
program goals are left out for some sites, the nature of the relationship between the 
program and the evaluation, and the selection of specific measures to operationalize 
the program goals. These differences across sites may serve as barriers to cross-site 
comparisons of program effectiveness, particularly if these sites must coordinate with 
a national evaluation team whose main goal is to assess overall program outcomes. 

Another major issue affecting the results of multi-site evaluations is differences 
in client risk levels. As noted earlier, it is essential that any evaluation, most 
especially a multi-site one, gather data about client risk levels so that analyses can 
adjust results to account for client differences in each program in the evaluation. An 
additional problem for multi-site evaluations is the pattern of client attachment and 
departure from the program. When some programs try to attract youth for extended 
periods of years, and others consider their task done in a four to six week period, the 
probable impacts of the programs are so different that one may not want to include a 

them in the same evaluation. 

Identifying Appropriate Comparison or Control Groups 

To demonstrate that a program made a difference, outcomes for program 
participants must be compared to something— either to outcomes for some comparison 
group of non-participants, or to the participants’ pre-program behaviors, as their own 
controls. Own-control designs work best when participants have exhibited a stable 
characteristic over a number of years (e.g., average annual days of hospitalization), 
and that stability would be expected to continue if the program does not intervene to 
change it. These designs are not a good choice for youth, since youth are in too much 
flux for any pre-program characteristics to be stable, or to be expected to stay stable 
during the measurement period in the absence of the program. This leaves various 
comparison or control group options. 
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We now discuss the choice of potential comparison groups within the larger 
issue of the choice of evaluation designs, assuming we want to conduct an outcome 
evaluation. Choice of comparison groups will be examined in light of the specific 
criteria for service integration programs for at-risk youth, and with reference to how 
the choice of design, and particularly the choice of comparison groups, will need to 
take into account the special features of these programs. 

In general, the choice of an outcome evaluation design is guided by the need to 
maximize both internal and external validity, while maintaining the integrity of 
program operations, allowing the program to operate with as little disruption as 
possible, and hopefully producing results that are meaningful and useful to the 
program as well as to other policymakers. There is an inherent trade-off between 
internal and external validity. Internal validity refers to the validity of conclusions 
about whether a given treatment was causally related to the measured outcomes, 
while external validity refers to the generalizability of the evaluation findings across 
subjects, time, and settings (Mark and Cook 1984). Interned validity is improved when 
one can control or explain away all possible extraneous influences on the relationship 
between treatment intervention and measured effects. But when one attempts to 
control for all potential sources of variation, the generalizability of the findings 
(external validity) usually suffers. Internal validity is maximized through the choice of 
a sound evaluation design with an appropriate control group. External validity can be 
improved by using random selection, or by deliberately sampling for heterogeneity, or 
by selecting prototypes or "modal instances" of specific people, settings, and times for 
inclusion in the evaluation, or by specifying the population of interest. 

Ideally, one would want to randomly assign participants to "treatment" and "no- 
treatment" groups. Random assignment to groups will, on the average, assure 
comparability of these groups at the outset, thereby ruling out selection bias. 

However, random assignment of youth to program and no-program groups is rarely 
possible (Mark and Cook 1984), due to: 

* Resistance by program personnel; 
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• The likelihood that the random assignment process will be compromised due to 
refusals or participant attrition; 

• Reactions of participants; 

• Operational errors in the randomization process which would result in 
nonequivalence of the groups. 

Some features of comprehensive service integration programs for at-risk youth also 
argue against random assignment to groups, such as the focus on individualized 
assessment of a youth’s service needs and a coordinated, case management approach 
to meeting these needs. Random assignment would prevent both of these service 
functions from occurring, since assignment to groups would not be based on meeting 
the needs of the individual but rather on meeting the needs of the research. 

Without random assignment to groups, we must investigate alternative means 
of ensuring that the planned "treatment" and "comparison" groups are equivalent at 
the outset of the evaluation study. Some alternatives to consider include the use of a 
matching strategy in which comparison group subjects are matched to similar 
intervention group subjects on a variety of key criteria such as risk status, indicators 
of risk, age, gender, etc. Two types of matching are possible; proportional and one-to- 
one. In proportional matching, the average mix of participant characteristics in the 
intervention and comparison groups are matched, so that, in general, the groups are 
deemed to consist cf similar types of individuals. This type of matching can be made 
more effective by correcting statistically during analysis for selection bias. A more 
effective but time-consuming and costly approach is one-to-one matching, in which 
comparison group participants are meticulously selected because they possess the 
same set of characteristics as each member of the intervention group. Finally, one 
can match "populations," selecting treatment schools and comparison schools (e.g., 
Zabin et al. (1986), or treatment housing projects and comparison housing projects 
(Schinke, Orlandi and Cole 1992), or treatment communities and comparison 
communities (Polit, Quint, and Riccio 1988), where the population characteristics of 
the comparison sites are matched as well as possible to those of the treatment sites. 



There is also a form of statistical matching which occurs once all study 
participants are recruited. Here, the analysis of post-treatment scores includes only 
those intervention and control subjects with similar pretest scores. These matching 
schemes are all highly flawed and their unreliability may lead to an underadjustment 
for existing group differences which might result in a pseudo-treatment effect (Mark 
and Cook 1984). In addition, since the central characteristic of the program 
participants for these comprehensive, service integration programs is their at-risk 
status, any matching of youth according to risk status will hinge on the accuracy of 
these risk measures. At present, most measures of risk (in our sense of the term 
which includes risky environments and risk markers) tend to suffer from too many 
false positives; they identify youth as high risk although the youth never exhibits later 
dysfunction or problem behavior. 

In a "quasi-experimental" approach, the individuals typically self-select into 

V 

treatment or are assigned to treatments by program personnel. In the target 
programs, the case manager will generally determine the set of services each youth 
will receive. In some programs, it will also be true that youth participate in some 
recreational or developmental activities whether or not they receive "services" from a 
case manager. In fact, these programs are likely to see the activities as "the program," 
and the services as important but for use "as needed." 

One p ossibility often used in evaluations is the pretest-posttest, nonequivalent 
groups design. In this design, measures are taken prior to the intervention and then 
following the intervention (the posttest) for both a treatment group and a 
nonequivalent control group. Usually, the nonequivalent control group will consist of 
either no-treatment or some minimal level of service provision. An obvious choice for 
a potential control group would be youth from the community who are not involved in 
the program. The youth may be drawn from the same community or, in cases where 
the program is particularly well-integrated in the community, the control group youth 
may come from another community which is matched according to its social 
indicators and quality of life. 





94 



The primary threat to the internal validity of the pretest-posttest nonequivalent 
groups design is "selection-maturation." This problem occurs when the observed 
effect may be due to the participants in the two groups maturing or developing at 
varying rates, independent of program effects. This problem often occurs in 
evaluations of compensatory education programs, when the treatment group consists 
of the most educationally disadvantaged but the control group consists of individuals 
whose academic performance and rate of increase in school achievement is higher 
than that of the more disadvantaged group. Since the control subjects are maturing 
at a faster rate than the experimental group participants, any change due to 
treatment would be obscured by the maturational change in the control group. 
Although attempts should be made to ensure that individuals in the intervention and 
control groups are similar on key characteristics, we have already pointed out how 
difficult this is. Without the guarantee of equivalence that random assignment brings 
to the evaluation, selection bias rather than treatment effect may account for any 
observed impacts. 

A second potential comparison strategy, available with large programs, might 
be to use sub-groups of the intervention participants. Each group could differ in 
planned ways, such as the mix of services or the level of intensity of services they 
receive. The greater number of services offered by comprehensive, service integration 
programs for at-risk youth ensures that there will be a wide range of intervention sub- 
groups that might make plausible comparison groups. But perhaps the range will be 
too wide, and too few individuals will have received similar services. In order for this 
to work as an evaluation design, the program would have to create systematic 
variations in its offerings to clients independent of client need. It would not work as 
an evaluation design if the analytic sub-groups were composed after services were 
provided, because then the level of presenting service needs (client risk levels or client 
difficulty) would confound the composition of the sub-groups. 

If the sub-group approach is done properly, the comparison would involve 
identifying the mix of services that provided the greatest change from pre- to post- 
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treatment. It would thereby answer a slightly different evaluation question from the 
one answered by a no-treatment control group. When using a no-treatment control 
group, we are essentially asking "is the target experimental program better than no 
program?" But with a set of planned comparison groups, we are assuming that the 
answer to the former question is affirmative and we want to now determine which mix 
of services or which specific services are the "active ingredients" of the program. 

If one were not as interested in answering the question of treatment impact, 
one could design a study which used the case management feature of the programs to 
advantage by first studying and identifying those factors which predict the services an 
individual will be offered. Knowledge of the factors used by case managers in 
determining service needs might become a useful focus of the outcome evaluation, 
rather than strict attention to treatment effects per se. By tying knowledge of 
selection factors to service mix, and then comparing clients who received different 
clusters of services on their degree of change over time, one could obtain useful 
information about both treatment effectiveness and treatment processes and design. 

A cautionary note here is that by using a less rigorous design, causal inferences 
regarding change over time may become tenuous. In other words, it will be harder to 
justify the conclusion that the changes were due to the type of treatment received. 

Another potential method for generating control or comparison groups is to 
consider cohorts as controls. Schools, for example, advance children from one grade 
to another, so that children from one group are together from one grade level to the 
next. Cohorts are groups of individuals or other units that follow each other through 
both formal (e.g., schools) and informal (e.g., families) institutions. Since cohorts tend 
to be similar in background characteristics, they make it more likely that groups will 
be equivalent at the outset of the evaluation. In an evaluation of comprehensive 
service integration programs for at-risk youth, particularly those located or linked to 
schools, we might use as control groups the children from earlier and later grades and 
then follow all younger and older cohorts across time. If, for example, the school- 
based program resulted in decreased teenage pregnancies, we would expect to see the 
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strongest results among those cohorts who were exposed to the intervention the 
longest. This is essentially the pattern reported by Zabin et al. (1986). Similar 
benefits, such as preventing school dropout, might be demonstrated by including 
cohorts of children from Grades 5 through 1 1 and then following them over time until 
the youngest children reached the highest grade level. 

History is one of the major threats to the internal validity of the cohort-as- 
control design, particularly since participants’ experiences in the program may change 
over time as a function of different policies, funding levels, and the typical "fine- 
tuning" which occurs with most innovative programs. Thus, different cohorts might 
be exposed to different experiences and influences, depending upon which year they 
became eligible for the program and how old they were when they received these 
services. In the end the cohorts may no longer be equivalent, due to these differences 
in program experiences. Given the innovative nature of many of the services we are 
interested in, cohort designs may only be appropriate for comprehensive service 
integration programs for at-risk youth once the program is relatively well-defined, has 
been operational for some time, and has been relatively "set” in terms of the nature 
and intensity of the services offered. 

Finally, another source for comparison data involves using the treatment group 
as its own control in a removed treatment design. By planning the introduction of 
specific services followed by their removal, and carefully measuring the effects on 
participants at each transition point, a case may be made for the benefits of particular 
treatment units or services. This design has the advantage of also helping to identify 
"active ingredients" of the program, but there are many shortcomings. There must be 
some reason to expect to see changes in program participants shortly after the 
introduction and again the removal of a service component. Generally, the prevention 
focus of these services makes it unreasonable to expect that one would find such 
short-term results. In addition, an ethical problem arises if treatment removal is 
frustrating for the participants or reduces their level of commitment to the program. 
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The internal validity of all of the designs just discussed could be enhanced by 
using multiple measurement points in assorted time series designs such as multiple 
baseline measures, simple or interrupted time series, or other forms of longitudinal 
procedures. These could be added on to a multiple comparison group pre-test- 
posttest design to rule out threats such as history or maturation. 

One of the main problems with multiple testing designs, in which treatment or 
comparison group individuals serve as their own controls over time, is that attrition 
will increase over time and may nullify any attempts to establish equivalence between 
groups. As noted earlier, subject attrition may be extensive in programs for at-risk 
youth, since the key task of the program is to "hook" the youth and get them 
committed to the program. While youth may be committed initially, they are also 
subject to many influences which may draw them away. A typical pattern for these 

youth might be that they come to the program for a short while but then stay away for 

i 

a period of time before returning. This would make it difficult to determine exactly 
how much of the program or intervention the youth actually received, and reduce the 
chances of treatment impact. It might be possible to group participants according to 
the levels of service received and then compare outcomes over time. Treatment impact, 
might be observed if those who received more services showed greater changes than 
those who received lower levels of service, but there is still the problem of ruling out 
selection bias. Clients who received more services might have been more motivated, 
or might have been more troubled, or might have been different in other ways, so that 
characteristics of the individuals affected self-selection and treatment exposure to 
account for the positive changes. 

Reducing Attrition at Follow-up 

Any evaluation interested in learning whether a program makes a long-term 
difference for clients faces the problem of recontacting all the people who participated 
in the program during some follow-up time period such as one year after program 
entry. Researchers often cannot find a significant number of participants in this 
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follow-up effort, and this problem is known as attrition from follow-up. Next to the 
problem of defining the comparison group, maintaining follow-up participation will be 
the hardest task, but the most critical one for producing convincing results. 

Initial sample sizes must be large enough to assure that the final follow-up 
sample will have enough respondents to test hypotheses even with expected attrition. 
Excessive attrition (a final sample size that is too small) will reduce the power of 
statistical tests, which in turn will increase the risk of Type II error (finding no effects 
when in fact there are effects). 

Another consequence of follow-up attrition is that the intervention and 
comparison groups may no longer be comparable at the time of follow-up. For 
example, if members of one group are harder to find than members of the other group 
for systematic reasons (e.g., spending more time in jail), results obtained through 
follow-up on important variables will be biased, and any conclusions drawn will be 
unreliable. Follow-up methods must assure, to the extent possible, that people lost to 
follow-up do not differ in important ways from the people for whom data were 
collected. At the very least, biases introduced by attrition should be examined. 

Maintaining contact with both treatment and control/comparison groups at 
follow-up is essential for results to be credible. Often not enough resources or 
ingenuity is put into this effort, particularly if the follow-up is left to program staff on 
top of their regular duties, rather than assigned to people whose only job it is to 
complete follow-ups. There are a variety of methods for reducing attrition to follow- 
up, including multiple time period tracking, paying clients, offering a lottery or prize 
for keeping in touch, and securing the cooperation of program staff (who often will 
continue to have contact with the attrited cases but may be too busy to tell the 
evaluators). Researchers can contact other family members and friends, schools, 
drivers license bureaus, credit bureaus, unemployment office records, and even debt 
collection or detective agencies. Some evaluations have had particular success in 
tracking down the program participants for follow-up interviews. For instance, the 
Perry Preschool Project conducted follow-up interviews with participating children five, 
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ten, fifteen and twenty years following the intervention (Berreuta-Clement et al. 1984), 
achieving response rates of 90-95 percent. Many of these longitudinal investigations 
provide interesting case studies of methods for reducing attrition at follow-up. 



SUMMARY: RELEVANCE FOR SITE VISITS 

This chapter has discussed evaluation issues specific to youth-serving 
programs, to service integration efforts, and to general issues of conducting 
evaluations with community programs. It has also discussed issues surrounding the 
selection of appropriate outcome measures and appropriate comparison or control 
groups. 

Anyone potentially interested in evaluating programs for at-risk youth might 
explore a number of these issues. The history and community context of the program 
will be particularly important to understand since it will help determine the scope of a 
potential evaluation. One would also want to understand as much as possible about 
existing SI linkages and arrangements and how these came about. Given the 
importance of program goals in determining what outcomes to measure, one would 
want to talk with program and community informants at length to understand what 
the program is trying to do and how it sees its present approach as advancing those 
goals for its clients. In addition to a focus on potential evaluability, it would be 
important to catch the "gestalt" of the program and understand the choices faced by 
programs that try to serve at-risk youth. One would also explore program goals and 
program approaches in order to understand what programs think youth need and 
how they feel they have to go about attracting and supporting them. 

The potential evaluator should be interested to learn whether the programs 
have participated in any evaluations, or whether they have thought about evaluation. 
Their attitudes toward evaluation should be assessed, and discussions should address 
what an evaluation would have to look like in order to be compatible with their 
program operations. It is important to examine any record-keeping systems they may 
have, and look at the mechanisms they have developed for recording participation and 
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tracking user/client service needs and service receipt. It is also important to discuss 
the issues of how "clients" are defined and how the programs try to affect families and 
neighborhoods, if they do. If comprehensiveness is a focus, one would examine what 
the programs mean by "comprehensive" and the various routes they have taken to 
achieve access to services for their clients. Finally, one would explore with the 
programs their reasons for developing and participating in an SI network and what 
they think the network is doing for them and their clients. 

We conclude with a quick summary of issues covered and the implications of 
our discussion for designing evaluations. 
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EXHIBIT 3.T: SUMMARY OP EVALUATION ISSUES AND IMPLICATIONS 



ISSUE 


IMPLICATIONS 


Defining the 

participant 

and unit of analysis 


Follow participants from point of entry rather than trying to identify 
"termination;" design evaluation to include components that account for all 
major program activity, even if much is heavily preventive and definitions of 
"who is a client" vary with different components. 


Effects of varying 
client risk levels 


Collect information at intake reflective of client status on multiple risk 
indicators, including antecedents, markers, and behaviors; include in evaluation 
design the ability to determine whether client risk levels affect selection into the 
program, and services offered once in the program; expect to use risk level 
indicators as one element in anafysis of impacts. 


Documenting the 
program, for impact 
analysis and for 
process analysis 


Any good evaluation, process or outcome, must include the capacity to 
document what each client gets, including services and activities, whether 
delivered by the program itself or through the program's network of referrals and 
inter-agency associations. For process anafysis with SI efforts it is also 
important to document the process through which services were delivered. 


Non-client outcomes 
of interest 


If the evaluation is interested in system change and its impact plans must be 
made to assemble and analyze evidence of such change. It is also important to 
be able to distinguish the effects on client outcomes of SI from those of 
comprehensiveness, and the effects of SI on comprehensiveness. 


Conducting 

evaluations 


In general, outsiders should conduct evaluations, with extensive interaction and 
participation of program personnel. Programs must be ready for evaluation, in 
the sense of willingness to cooperate and interest in the results and also in the 
sense of having adequate systems in place to collect and process the necessary 
data. 


What outcomes and 
impacts to measure 


Outcomes selected for measurement may pertain to individual clients, their 
families, other members of the community, or the neighborhood as a physical 
space. For multi-site evaluations, outcomes selected must be relevant to all 
programs and should also reflect the better part of their goals and activities. 


Comparison /control 
groups 


Evaluations are significantly more convincing when they include a control or 
comparison group. The most likely design for programs of the type examined 
here is a quasi-experimented design with comparison groups selected from 
similar communities that do not have the program. Such a design is not as ideal 
as random assignment, but is a good deed better them no compeulson group at 
edl. 


Reducing attrition at 
follow-up 


Next to including em appropriate compeulson group, nothing is more importemt 
to the integrity and persuasiveness of an evaluation than maintaining adequate 
levels of contact for follow-ups. Given what will edready have been invested in 
the eveduation, it is worth considerable trouble emd expense to etssure low 
attrition to follow-up. 
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CHAPTER 4 



CONDUCT OF THE SITE VISITS 



Having completed reviews of risk prevalence among youth and the ability of 
traditional single-focus programs to help youth avoid serious risk, we turned to an 
examination of what might be expected from service integration efforts directed toward 
this population and of issues that might arise in undertaking evaluations of more 
comprehensive programs for at-risk youth. To ground the results of these reviews in 
the reality of program operations, we considered it essential to visit a variety of 
programs that try to deliver comprehensive services, using service integration as at 

least one mechanism for increasing comprehensiveness. Thus, site visits were 

» 

included in the study design to achieve several of the project’s objectives: 

• To understand the full range of program configurations and options for 10- to 
15-year-olds, including the programs’ sense of their mission or purpose: 

• To understand the reasons behind these programs’ choices among certain 
program design alternatives (e.g., whether to emphasize "activities" or 
"services;" whether to concentrate on prevention or on treatment; whether to 
adopt a focus on youth, on youth plus their families, on families in general, or 
on the total neighborhood; whether to strive for comprehensive service delivery); 

• To understand the relationship of these programs to their larger community, 
including both the program’s role in the service delivery network and network 
of supports for youth, and the program’s role in relation to other community 
institutions such as churches and community centers; 

• To learn what programs we believe are the benefits of a more comprehensive 
range of services, and what they believe are the benefits and drawbacks of 
service integration through collaborative arrangements with other agencies; and 

• To gain a sense of the readiness and willingness of programs of this type to 
participate in evaluations, and what types of evaluations they might be open to 
(or have already been involved in). 



SEARCH PROCEDURES AND CRITERIA FOR SELECTION 

The first step in the process of selecting programs that provide comprehensive, 
integrated services to at-risk youth between the ages of 10 and 15 was consultation 
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with several experts in the field of youth and youth services. We were looking for 
programs that met the following criteria: 

1. The program serves clients between the ages of 10 and 15; 

2. The program conducts comprehensive, individualized needs assessments for 
individual youth; 

3. The program uses the needs assessment as the basis for service planning/ case 
management; 

4. The program has developed formal, institutionalized inter-agency linkages (e.g., 
resource sharing, case management); 

5. The program conducts standard follow-ups with agencies to which referrals Eire 
made to ensure accountability. 



We contacted experts at the following organizations and asked to recommend 
programs that fit these criteria: 

• Charles Stewart Mott Foundation 

• Stuart Foundation 

• Carnegie Council on Adolescent Development 

• National Governor’s Association 

• Boys and Girls Clubs of America 

• Girls Inc. 

• Big Brothers/Big Sisters of America 

• CMdren’s Defense Fund 

• MathTech, Inc. 

• National Center for Service Integration 

• California School-based Service Integration Project 

• National Center for Children in Poverty 

• National Network of Runaway and Youth Services 

• Erickson Institute 

• National Resource Center for Youth Services 



In addition, two publications (Partnerships for Youth 2000: A Program Models 
Manual , 1988; The Future of Children: School-Linked Services. 1992) provided 
program descriptions and contact information. Based on these resources, we 
assembled an initial list of more than 30 programs to consider for five site visits. 

HHS guided the process of shortening this initial list with several preferences. 
Since this project is exploratory and meant to generate new information, programs 
which have already received a great deal of publicity or for which evaluation results 
have been reported were taken off the list (e.g., the Door, New Beginnings. New 
Futures, and the New Jersey School-Based Youth Services Program). Two lesser - 



known programs were ruled out because the Office of the Assistant Secretary for 
Planning and Evaluation (ASPE) at HHS is funding them as service integration 
demonstration models. Since ASPE is familiar with these two programs, we 
considered further inquiry unnecessary. We eliminaed school-based health clinics 
from our search as well, since this program model is already quite well known and 
has been the subject of several evaluations. 

These deletions left 20 programs on the list. We conducted telephone interviews 
with program directors from each of these programs. The interviews ranged in length 
from ten to forty minutes and covered clientele, intake and other procedures, 
communities served, and inter-agency connections and arrangements. We eliminated 
three programs from our list because they serve few or no youth ages 10 to 15. 

Among the remaining candidate programs, almost all met at least three of our five 
criteria. Formal inter-agency linkages and tracking of services were the criteria most 
often left out. 

Since only a handful of programs met all five criteria, some flexibility was 
introduced into the final selection process. For instance, as long as a program was 
able to meet a youth’s service needs through informal ties with other agencies, formal 
inter-agency agreements were not required. However, to remain on the list a program 
had to offer individualized service planning, either by case management or counseling. 
In addition to the five formal program criteria, final program selection was also based 
on a desire to have the final set of programs represent a good mix across geographic 
regions, serve youth of diverse racial or ethnic backgrounds, and use a variety of 
program models (e.g., residential, school-based and community-based; case 
management, treatment, activities/enrichment; crisis, medium-length, long-term). 

We presented HHS with a list of the eight most promising examples of lesser - 
known comprehensive, integrated service programs for at-risk youth aged 10 to 15. 
Based on HHS recommendations, we dropped two California programs. Two programs 
that had been excluded because they met relatively few of our five criteria were 
reintroduced because they represented interesting program models (Big Brothers/Big 
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Sisters of Greater Miami and Project Step Ahead in the Bronx). We also substituted 
Teen Connections in the Bronx for Teen Connections in South Dakota. In addition, in 
Nashville and Miami we added one more program each to include programs serving a 
predominantly African-American clientele, since this population was underrepresented 
in the array of other sites recommended. 

SITE VISIT PROCEDURES 

We contacted each program by telephone to arrange the timing of the visit and to 
establish a schedule of interviews prior to our arrival. Before going on the visit we 
also obtained each program’s brochures, other descriptive literature, and in many 
cases copies of local newspaper stories about the program and the youth it serves. At 
most sites the program director or assistant director was available throughout our 
visit to facilitate meetings and arrangements and answer many formal questions 
about the program’s history, service configuration, mission and goals, future 
directions, and funding. 

We conducted interviews with program directors, directors of individual program 
components, line workers with direct youth contact, youth, families of youth, mentors 
or volunteers where relevant, representatives of agencies with which the program has 
formal or informal linkages, and members of the program’s Board of Directors where 
relevant. Exhibit 4. 1 displays the people with whom we conducted interviews and the 
topics covered with each. 

In addition to conducting interviews, we also collected and examined a variety of 
program documents. These include the program’s most recent budget documents 
(sources of income and cost centers/expenses); all intake forms used for case 
management; available program statistics; any evaluations or reports containing 
indications of program impact; miscellaneous other documents such as program 
brochures, newsletters, curriculum modules, community education materials, and 
similar material as available. 
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OVERVIEW OF PROGRAMS VISITED 



We planned to visit a total of ten programs in six locations that represent a mix of 
geographic location, target population, and program model. One program (Project 
Step Ahead in the Bronx) was ultimately dropped from the list because its funding 
expired and it closed its doors before our field work began. Each of the remaining 
nine programs serves individual youth, youth and their families, or a whole 
community. Exhibit 4.2 gives an overview of the programs’ basic characteristics. To 
summarize, the nine programs can be characterized as follows: 

• Age Range -- The proportion of 10- to 15-year-old clients served by the programs 
ranges from about 50 percent to 100 percent. 

• Sex -- One of the programs serves only girls; the remainder serve both boys and 
girls but tend to have more boys. Our impression is that the more the programs’ 
clients come through referrals from formed agencies such as the courts or child 
protective services, the higher the proportion of boys. 

• Race /Ethnicity -- Our selection criteria resulted in a deliberately varied group of 
programs with respect to the race or ethnicity of clients served. Two programs 
serve almost entirely African-American youth, two serve mostly white youth, one 
serves mostly Hispanic youth, two serve a mixed group of Hispanic and African- 
American clients; and two have very ethnically mixed groups of users. 

• Focus of Activities/Services -- Three programs focus their efforts mostly or 
exclusively on the youth themselves, but may assist a youth’s family if it becomes 
apparent that help is needed; three programs focus on youth in some of their 
activities and place a heavy emphasis on involving the families of youth in other 
components of the program (e.g., for "caseload" clients); three programs have 
some activities mainly for youth, some services that involve youth and their 
families, some offerings for any interested community member, and an 
overarching goal of changing and empowering the whole community. 

• Program Model — The nine programs include one mentoring program, one 
program focusing on a geographically-defined community, one program operating 
almost entirely in the schools, three programs that operate in both schools and 
the community, and three community-based programs. Five of the programs use 
case management, but these programs vary in the proportion of their youth 
clients who receive case management. Three programs provide crisis -oriented 
short-term services. 

• Meeting Selection Criteria -- Five of the programs visited met all five of the 
criteria established for program selection (see above). One program met four 
criteria, two programs met three criteria, and one program met only two of the 
criteria. The criterion most often missing was the ability of programs to obtain 
and record successful service delivery by agencies to which they had referred 
their clients. Also, several programs did not have formal inter-agency linkages to 
facilitate service integration (although they did have informal arrangements and 
understandings with other agencies). 



EXHIBIT 4.1 

INTERVIEW TOPICS AND INTERVIEWEES 
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CHAPTERS 

PROGRAM DESCRIPTIONS 



During site visits we worked to gain a basic idea of each program’s scope of 
activities and services. We also wanted to learn about the program’s history, who it 
serves, its role in the larger service network of its community, and its involvement in 
evaluation activities. We also directed many of our specific inquiries toward 
understanding how each program handled several issues we think of as cross-cutting. 

This chapter presents descriptions of the nine programs we visited. Each 
description is organized into the following sections: 

• Brief history. 

• Current mission, goals, objectives. 

• Service configuration. 

• Current clientele/users. 

• Type and makeup of SI network. 

• Funding sources. 

• Evaluation. 



After the reader gains an idea from these descriptions of what each program does and 
how it relates to its larger community, we take up a discussion of the important cross- 
program issues in Chapter 6. 



THE BELAFONTE-TACOLCY CENTER, INC, 



Executive Director: 
City/ State: 

Phone Number: 



Yvonne McCullough 
Miami, Florida 
( 305 ) 751-1295 



Brief History 

Belafonte-Tacolcy Center began in 1967 as a grassroots youth-serving 
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organization founded by a group of young men and was originally known as The 
Advisory Committee of Liberty City Youth (TACOLCY). Its core youth-serving 
programs stem from this time. The name was changed to Belafonte-Tacolcy Center in 
1969 to commemorate a donation to the new facility by singer/ actor Harry Belafonte. 
In 1970 it incorporated as a private non-profit organization. The City of Miami owns 
the present Tacolcy facility, which was built in 1969, and leases it indefinitely to 
Belafonte-Tacolcy Center. 

The Belafonte-Tacolcy Center’s core mission has not changed significantly over 
the years, but new activities and programs have been added over time in response to 
community needs. The original youth programs included recreational activities, 
enrichment groups, a summer youth employment program, youth leadership groups, 
and a cultural arts program. These are still the core activities at Belafonte-Tacolcy. 

In general, programs added later focus on preventing specific problems such as 
alcohol and drug abuse, academic failure, gang membership, HIV/AIDS, and youth 
unemployment, and on promoting child and adolescent development. 

After the 1968 race riots, which took place largely in the Liberty City area 
where Belafonte-Tacolcy is situated, the second Executive Director of Belafonte- 
Tacolcy, Otis Pitts, started the Tacolcy Economic Development Corporation (TEDC). 
TEDC is not a program offering services for youth. Rather, TEDC began with the goal 
of rebuilding the neighborhood, and continues to contribute to its maintenance and 
further development through large projects such as building apartments and 
shopping centers. Its rationale is that economic development creates jobs, improves 
neighborhood safety, and makes available more affordable housing, thus enhancing 
the local economy. Belafonte-Tacolcy Center owns a shopping center developed by 
TEDC, and a portion of the profits get funnelled back into the Center to support its 
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youth programs. 

Belafonte-Tacolcy has added several program components for youth since 1981. 
The Community Outreach Intervention and Cultural Appreciation Program (COICAP) 
was added in that year as a school dropout and juvenile delinquency prevention 
program. The school-based Drug and Alcohol Abuse Prevention Program was 
developed at approximately the same time. Belafonte-Tacolcy worked with a number 
of other community agencies to start a Haitian Outreach Center in the late 1980s to 
provide outreach and one-site access to many services for newly arrived Haitian 
immigrants. The most recent program additions include a community-based 
HIV/AIDS awareness program (1989) and a Anti-Gang Program (1990). 

The histoiy of Belafonte-Tacolcy also reveals strong continuity between the 
initial cohorts of youth served by the Center and today’s youjth. Many of the 
program’s current leaders and managers were themselves youth participants in 
Belafonte-Tacolcy when it first started in the 1960s. 

Current Mission, Goals, Objectives 

The overall mission of Belafonte-Tacolcy is to provide "diversified services to 
children, youth, and young adults aged 2 V6 to 26 years that can allow them to become 
responsible, productive citizens." The goals the program has set to carry out this 
mission include: increasing social functioning, building leadership skills, and fostering 
healthy adolescent development. The program emphasizes comprehensive services 
that span a wide developmental continuum, from pre-school-aged children to young 
adults. All phases of a youth’s development are addressed through a mix of 
educational, counseling, recreational, vocational, and leadership training activities. 
The Center functions as a community center or "clubhouse" with a strong prevention 
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focus. 



Preventive interventions, typically delivered in group settings either on-site or at 
schools, focus on drug and alcohol abuse, school dropout, educational development, 
juvenile delinquency, gang membership, and HIV/AIDS awareness and health 
enrichment. The physical facility provides comfortable, aesthetically pleasing 
surroundings where children and youth can spend a portion of their time each day. 
Belafonte-Tacolcy also collaborates with other community agencies, including mental 
health centers, health crisis counseling, food banks, recreational centers, churches, 
and local private groups such as the Private Industries Council (PIC). Its programs 
primarily target the children and youth themselves, and secondarily target parents, 
who participate in individual counseling, group meetings, and workshops. 

Service Configuration 

Specific program components are delivered at the Belafonte-Tacolcy Center and 
at a number of elementary, middle, and high schools. There are a variety of programs 
aimed at younger children, including a meal program, day care and after-school care, 
and other child development enrichment activities. There are also a number of 
activities aimed at older adolescents and young adults, including the Stay-in-School 
project, the Youth Vocational Training and Employment Opportunity Program, and 
adolescent development enrichment activities. Since the focus of this report is at-risk 
youth aged 10 to 14, we will deal primarily with activities geared toward this older 
group. 

The basic model of all prevention activities at Belafonte-Tacolcy consists of a 
group workshop typically delivered within a classroom setting, in which the Belafonte- 
Tacolcy staff member presents a formal curriculum and at the same time identifies 
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more serious problems in specific individuals. Those youth identified in the 
workshops as being at high risk are then offered more intensive services at Belafonte- 
Tacolcy, consisting of comprehensive needs assessment, academic tutoring, peer 
counseling, and parental support. The parent or youth may also be referred to 
outside services including mental health centers, health crisis counseling centers, 
child welfare agencies, or other community-based groups. Belafonte-Tacolcy uses this 
basic model in the Community Outreach Intervention and Cultural Appreciation 
Program (COICAP), the Drug and Alcohol Abuse Prevention Program, the Anti-Gang 
Program, and the HIV/AIDS health awareness program. A variation of this basic 
model involves identifying the high risk youth through their participation in 
recreational and sports development programs at Belafonte-Tacolcy and then referring 
these youth for comprehensive assessment and more intensive services if required. 

The central, and longest-running, prevention activity at Belafonte-Tacolcy is the 
Community Outreach Intervention and Cultural Appreciation Program (COICAP). This 
is a combined crime prevention and school dropout prevention program for youth 6 to 
18 years of age. Most youth are referred by school counselors and teachers or the 
juvenile justice system, typically the courts. Some youth are also "walk-ins": they ask 
for help with school and, after assessment, become eligible for all program activities. 
COICAP offers an extensive psychosocial risk assessment, including a home visit, 
which is followed by an individualized treatment plan. Most plans include after- 
school educational and developmental workshops that emphasize self esteem building; 
anxiety management; decision making; problem solving; and academic skills 
enhancement consisting of tutoring, diagnostic assessment, monitoring progress, and 
working with school teachers. Finally, parents of COICAP youth become involved in 
parenting skills development workshops and family counseling if appropriate. Youth 
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in COICAP also participate in a variety of field trips, including Outdoor Challenge, a 
wilderness stress/challenge program. 

The Drug and Alcohol Abuse Prevention Program is similar to COICAP but 
works with youth at risk for becoming involved in drugs. The program attempts to 
provide a comprehensive support system through a combination of school-based 
enrichment workshops, supervised recreational activities, educational 
tutoring/enrichment, individual counseling, and home visits. Belafonte-Tacolcy staff 
provide workshops on drug abuse prevention within the classrooms at various 
elementary and middle schools. Youth identified in these workshops by Belafonte- 
Tacolcy staff as requiring further assistance are then referred to the Center itself. 
There they may receive a more comprehensive assessment of needs and more 
intensive services and activities that follow the overall model of service delivery at 
Belafonte-Tacolcy (group and individual counseling, workshops in esteem building 
and refusal skills, academic tutoring, parenting skills development workshops and 
individual parental counseling, and referral to other agencies). 

There are a variety of additional prevention and enrichment programs for youth 
in the 10 to 14 year age range. The Anti-Gang Program offers weekly developmental 
workshops to 20 4th, 5th, and 6th grade classes in two primary schools. The 
workshops are geared to children of former gang members and children who "hang 
out" with older gang members, and focus on building self esteem, stress management, 
drug education, academic monitoring, individual counseling, parental involvement, 
and assessments. 

The Belafonte-Tacolcy Health Enrichment Program provides HIV/AIDS 
education throughout the community (e.g., in schools, churches, beauty salons and 
barbershops, and other places where teens and young adults congregate). The core 
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Health Enrichment Program consists of five one-hour sessions presented to 
classrooms, church groups, or other assembled groups of youth or parents. The 
program is delivered through a variety of techniques including videos, concerts, street 
outreach, lectures, music, role plays, and discussion groups. In school settings, 
Belafonte-Tacolcy staff provide a school-board approved, in-class HIV/AIDS awareness 
curriculum to all students. Finally, Belafonte-Tacolcy operates a youth crisis 
telephone line, conducts an off-campus work-study program involving college 
students doing peer mentoring and tutoring, provides a summer youth employment 
and training program, and works with other agencies in the Haitian Outreach Center. 

Current Clientele/Users 

As noi.ed earlier, Belafonte-Tacolcy serves children, youth, and young adults 
spanning the ages of 2Vi to 26 years. All children, youth, and their parents living in 
Liberty City are eligible for the programs and activities. Specific programs are geared 
toward various age groups as follows: day care is offered for 2 1 / 2 - 5 -year-olds, after- 
school care is provided to children 6-13 years old, the Liberty City youth enrichment 
programs (including COICAP and other prevention modules) are targeted at youth 
from 6 to 18 years of age, and the Stay-In-School, the Outdoor Challenge, and the 
Youth Vocational Training and Employment Opportunities programs are aimed at 
youth 14 years of age and older. 

The general profile of youth participating in Belafonte-Tacolcy programs mirrors 
the sociodemographic makeup of the Liberty City area: approximately 75 percent are 
African-American, 20 percent are recent Haitian immigrants and refugees, and 5 
percent are Hispanic. Most of these children and youth live in poor single-parent 
households and are exposed to open-air drug selling, neighborhood crack houses, 



high rates of criminal activity, and frequent violence. In general, Belafonte-Tacolcy 
considers all children living in Liberty City to be at risk and thus eligible for any 
Belafonte-Tacolcy programs. 

Each component program of Belafonte-Tacolcy has its own target population 
within the general category of children and youth living in Liberty City. To participate 
in the COICAP program, youth must have been involved with te juvenile justice 
system and/or have displayed academic performance problems that indicate a high 
likelihood of dropping out of school. The Drug and Alcohol Abuse Prevention Program 
conducted in school classrooms involves the worker targeting an entire class that 
contains a high number of youth displaying poor school performance, hehavioral 
problems, truancy, and/or prior drug involvement. Classrooms are identified by the 
school principal and teachers in conjunction with the Belafonte-Tacolcy worker. 

During the in-class workshop the Belafonte-Tacolcy worker identifies individual 
children who require more intense preventive interventions. The Anti-Gang Program 
originally served mainly the children of former gang members, but slow recruitment 
led the program to expand to youth showing behavior problems in school (as identified 
by the school guidance counselor). The Health Enrichment Program featuring 
HIV/AIDS awareness goes to all youth attending Dade County schools located in the 
Liberty City area, as well as to parents and other members of the community through 
church groups and other street venues (e.g., beauty salons and barber shops). 



Type and Makeup of SI Network 

Belafonte-Tacolcy is involved in three linkage networks. The first involves 
informal liaisons with other agencies in the Liberty City community. Second, 
Belafonte-Tacolcy has relatively well-developed links to the area schools through the 
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Dade County School Board. Finally, Belafonte-Tacolcy is a partner with other 
agencies in two community development projects, the Haitian Outreach Center and 
the Tacolcy Economic Development Corporation. 

Belafonte-Tacolcy has established informal ties to a number of agencies in the 
community including mental health centers, health crisis centers, the Department of 
Social Services, food banks, recreational centers, the James E. Scott Community 
Association (a direct service agency), and the juvenile courts and juvenile justice 
departments. Staff also sit on a number of interagency councils within the Liberty 
City area, each of which addresses a specific community issue such as hurricane 
relief, drug and alcohol abuse, mental health, and youth problems. These informal 
linkages come into play when staff note that a youth has a particular need that 
another agency can meet. 

A highly developed set of links exists between Belafonte-Tacolcy and various 
elementary, middle, and high schools in the Liberty City area. These arrangements, 
formalized in written documents, involve drug abuse prevention, anti-gang, and health 
enrichment programs which Belafonte-Tacolcy delivers in school classrooms. In 
addition, youth already enrolled in the Stay-in-School Program may be released from 
classes for individual work on school grounds with a Belafonte-Tacolcy staff person. 
Belafonte-Tacolcy staff also frequently consult with school principals, vice-principals, 
guidance counselors, and teachers about individual youth. 

Finally, Belafonte-Tacolcy Center participates in the Haitian Outreach Center, a 
collaborative effort of the United Way, the Salvation Army, New Horizons, Legal 
Services of Greater Miami, and the Center for Haitian Studies. The Haitian Outreach 
Center is a comprehensive multi-service center that addresses many needs of Haitian 
immigrants such as helping parents register their children for school, conducting 
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workshops on how to deal with immigrant problems, and holding English as a Second 
Language courses. The Salvation Army donates space and United Way funds pay for 
staff salaries of workers out-placed from many community agencies (Belafonte-Tacolcy 
has a full-time staff person at the Center). 

Belafonte-Tacolcy has an established plan for dealing with outside agencies 
who want to work with the center or who want to reach the youth at Belafonte-Tacolcy 
to fill in service gaps with new projects that will involve new funding. Once Belafonte- 
Tacolcy and the agency agree to work together to obtain funding and develop the new 
service, they draft and sign a written agreement. Acting on the agreement is 
understood to be contingent upon receiving the funding to support the project. 
Funding sources for these collaborations typically consist of state or Federal agencies 
with competitive grant programs; Belafonte-Tacolcy serves as the fiscal agent for 
grants resulting from successful applications. If a grant application is funded a 
Belafonte-Tacolcy program manager and outside agency personnel establish a more 
polished version of the working concept and solidify working relationships between 
Center staff and an individual staff member at the other agency involved. During the 
last five years, Belafonte-Tacolcy program managers have followed this approach of 
gaining commitments from cooperating agencies before writing a grant application. 
Prior to that time they sometimes received money for new programs without having 
specific agency commitments to cooperate, and found they spent too much of the 
grant period just setting up the arrangements before program services could begin. 
Now, agreements are formalized before applying for funding with the official in charge 
of each cooperating agency. 



Funding Sources 

Belafonte-Tacolcy receives funds from local, state, and Federal funding entities 
and from the United Way; it has an annual 1992 budget of $1.6 million. The 
breakout by funding sources is: 27 percent from Federal agencies, 19 percent from 
state agencies, 38 percent from local sources, and 16 percent from the United Way. 
Approximately one-third of local funds come from private donations. In the past 
several years some major changes have occurred in the proportion of funds from each 
source. United Way funds have remained relatively constant over the past five years. 
But as a result of additional fundraising and new programs, the United Way share of 
Belafonte-Tacolcy’s budget has dropped from 48 percent five years ago to its present 
16 percent. United Way funds have also fallen slightly in the past year (1 percent for 
all community agencies receiving funds) due to problems in the local economy. Over 
the last five years, an increasingly high proportion of program funds in a greatly 
expanded total budget have come from short-term demonstration programs that 
provide support for a specified time period. 



Evaluation 

Program staff at Belafonte-Tacolcy maintain detailed records of client 
participation in activities, including information on client backgrounds, assessment of 
needs and risk status and some pre-post tests of drug knowledge or gang affiliation. 
Staff open files on individual children whenever a child participates in any of the 
recreation or prevention activities at the Center, whether located on-site, at schools, or 
in the community. 

The agency as a whole develops a three-year plan to describe the scope of 
current activities and plans for expansion. It is updated every year by the whole 
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agency. In its annual update, each program or activity specifies the numbers and 
types of clients it expects to serve, the numbers and types of activities it intends to 
present or accomplish, and when appropriate, outcomes. An example of outcome 
measurement is pre-post testing for classroom presentations and group activities, to 
see whether youth have moved in the desired direction with respect to skills and 
abilities, knowledge, and behaviors. The manager for each program prepares a 
quarterly action plan that describes the accomplishments of the previous quarter, 
plans for the next quarter, and how a shortfall from the previous quarter (if any exists) 
will be made up during the coming quarter. Belafonte-Tacolcy monitors progress 
toward quarterly and annual goals through a monthly client service data report 
developed by the Program Director. This report specifies the number of services 
provided, the number of clients served, and the overall units (in hours) of service given 
each month. Programs also use a goal oriented recording system to identify goals for 
each client (youth and parent or family) and specific objectives to achieve these goals. 
In addition, many of the recreational activity programs and the school-based 
workshops also keep records, primarily of attendance, to track a youth’s participation 
across a set of Center-based and school-based programs and activities. 

The Program Director indicated substantial enthusiasm and willingness to do 
more in-depth evaluation of the program and was particularly interested in assessing 
long-term effectiveness using a longitudinal design. She was willing to involve 
Belafonte-Tacolcy in a research and demonstration project even if it meant changing 
some of the program procedures to accommodate the research study. However, the 
program currently lacks the resources and capabilities to participate in more 
extensive evaluation research. The program generates many forms and reporting 
tables, but they do not appear to use a computerized database for data entry and 
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statistical reporting, and information across program components does not seem to be 
clearly summarized. At this point, senior management may not have enough time or 
expertise to develop a more systematic information system without requiring 
significant outside technical assistance and additional resources. 



Brief History 

Big Brothers and Big Sisters have served the Greater Miami community for 
approximately 35 years. These two organizations joined in 1972 to form Big 
Brothers/Big Sisters of Greater Miami, an affiliate of Big Brothers/Big Sisters of 
America. The agency is well-known within the community and offers its clients a 
diverse range of mentoring or "match" services that are closely monitored by social 
workers. Originally the program focused solely on its "core match" program-- 
matching interested children between the ages of 6 and 18 who live in single parent 
families with volunteers who serve as friends and role models. Since the sole criterion 
for participation (within specified age limits) is that the youth come from a single 
parent family, the program’s clientele are from diverse backgrounds and 
neighborhoods. 

Approximately five years ago the program hired its current executive director, 
Lydia Muniz. The executive director recognized the need to conduct strategic planning 
and surveyed the community and the agency’s major funder, the United Way, to 
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obtain perceptions of the organization. In reaction to the perception that the program 
did not respond to the changing needs of the community, the executive director 
created a think tank, including the program’s senior management team and several 
generations of board presidents, to develop a new strategic plan. The result was a 
three-pronged approach to running the organization that includes programming and 
service delivery, volunteer recruitment, and fund development. 

Big Brothers/ Big Sisters has an active, committee-driven board involved in the 
organization’s policy decisions. The board undertook significant restructuring 
approximately five years ago to implement the organization’s decision to take a more 
aggressive role in member recruitment. The organization decided to target parents 
and younger professionals in an effort to create a more diversified board of directors 
with the expertise to meet the community’s needs. Additionally, to maintain board 
diversity the board moved away from indefinite terms for its members to three year, 
renewable terms. The terms are staggered to maintain continuity within the board. 

The activities of the new executive director and the restructuring of the agency’s 
board of directors helped to refocus the program’s services. Within the past five years 
the number of specialized program offerings has grown significantly and interest has 
increased in targeting a wider range of potential volunteer groups (e.g., older adults 
for the intergenerational match program). 

Current Mission, Goals, Objectives 

The overall mission of Big Brothers/ Big Sisters is to support and enhance 
single parent families by providing volunteer friends and role models who will help 
children develop their full potential. The children served by Big Brothers/ Big Sisters 
are considered at risk because they come from single parent families. 



The key goal of the program is to "provide concerned, responsible volunteers to 
serve as friends and role models to youth from one percent of Dade County’s single 
parent families before these children "get into trouble." The relationships developed 
through Big Brothers/ Big Sisters have as a major objective: providing positive adult 
role models through companionship and feedback. The Big Brothers/Big Sisters 
relationship is also expected to build self esteem and teach the youth new skills. The 
program has an additional goal of forging community linkages and collaborative 
partnerships with other organizations to meet the diverse needs of their client 
population. 

Service Configuration 

Big Brothers/ Big Sisters is essentially a role-modeling agency. Social workers 
closely monitor the mentoring relationships and also provide case management to 
families in need of referrals for services (e.g., mental health, housing, income 
maintenance). All potential clients go through a comprehensive in-home assessment 
by a social worker assigned to the child. The assessment includes an application form 
(for both the parent and the child) and a home visit to gather information on the 
family’s histoiy, discuss the child’s interests and questions, and ascertain the type of 
volunteer match desired by the parent. Once accepted into the program, the child is 
placed on a waiting list until an appropriate match is found. The amount of time an 
individual remains on the waiting list varies depending upon the characteristics of the 
match participants (e.g., African-American boys take longer to match when they desire 
African-American male volunteers). 

Potential volunteers also undergo extensive screening and assessment. They 
attend an orientation at which they discuss their expectations and receive an 



application form. Orientations occur twice a month at either the central office or one 
of its satellites. Potential volunteers complete an application and a psychological 
profile (16PF) and undergo an extensive background check that includes a screening 
of references, police record, drivers license, and HRS child abuse registry. A social 
worker assigned to the volunteer also conducts an in-depth home visit that includes 
an exploration of the potential volunteer’s background and past experiences. 

Approved volunteers are put on a waiting list until the program can match them with 
an appropriate youth. 

Matches are made on the basis of the social worker’s professional expertise and 
take into account the preferences, interests, and characteristics of client, parent, and 
volunteer. Once matched, the social worker contacts both parties to assess interest in 
the proposed match. If accepted, the volunteer, social worker, youth, and parent 
attend a match conference at the child’s home to review and sign copies of the 
program’s rules and regulations. Within two months of the match the social worker 
convenes a goal setting conference to set yearly goals and objectives for the match 
with all participants. These vary depending upon the needs of the child. The typical 
first year goal is to establish a relationship with the child. 

Standard procedures also include an annual review to evaluate progress toward 
goal attainment, assess the viability of the match, and generate new goals. The 
program includes a formalized match termination process that either the parent or 
volunteer can initiate. The volunteer and youth must spend between three and five 
hours together each week and the volunteer must make a one year commitment. 

Each social worker must spend part of his or her time at satellite offices located 
throughout Dade County. This allows Big Brothers/ Big Sisters to maintain a 
presence throughout the community. Social workers contact each parent, volunteer, 
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and child on a set schedule according to the length of the match relationship. They 
provide support to the volunteer and may refer the parent or child for additional 
services if needed. Approximately 40 to 50 percent of the caseload requires some type 
of referral or linkage to a community agency. 

Big Brothers/ Big Sisters has a variety of match programs to supplement its 
core program and meet the diverse needs of its client population. The program 
offerings include: 

• Intergenerational Match Program. Since 1989, Big Brothers/Big Sisters has 
offered an intergenerational match program in which older adult volunteers 
(aged 55 and up) provide companionship and support to a child. This program 
is an extension of the core match program, and evolved out of requests from 
parents for a mentor who is more than just a "big brother." Greater Miami’s 
intergenerational match program is one of nine pilot sites for national Big 
Brothers/Big Sisters’ intergenerational program initiative. It is a small 
program, serving 6-7 matches. 

• Teen Connections Program. Teen Connections began in 1987 and provides 
female mentors for 10- to- 16-year-old girls who are at risk for teen pregnancy 
and drug use. Volunteers for this program receive additional training during 
their orientation and participate in quarterly support groups to discuss issues 
related to serving this special population. 

• Special Needs Program. Since 1990, Big Brothers/Big Sisters has offered a 
special mentoring program for mentally and physically disabled and 
developmentally delayed 5- 18-year-olds. This is the only program in which 
clients need not come from a single parent family. The program currently 
serves 5 matches. Its major goals include: independence, normalization, and 
the development of leisure time activities. 

• Juvenile Justice Program. The Juvenile Justice Program began in 1992 to 
provide mentors to qualifying juveniles (those who are beginning to show 
evidence of delinquency and may have been involved with the juvenile justice 
system). The major goal of this program is to provide one-on-one experiences 
to build the self esteem of this group of at-risk youth. The program targets 
teenage boys from a local middle school with many high-risk African-American 
boys. Each child in the program is placed in contact with three adults: 1) a 
volunteer tutor from a local college or university who spends 1-3 hours each 
week at the school with a child; 2) a Big Brothers/Big Sisters match, and 3) a 
mentor advocate to help the child deal with any difficulties within the school 
system (e.g., fighting, poor attendance). This program has an evaluation 
component that includes comparisons with a control group— a middle school 
with a similar profile. 
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• Project C.A.R.E.S. (Children’s Advancement through Recreation and Educational 
Services). Project C.A.R.E.S. is an educational and recreational program for 
children who are waiting to receive a match. It began in 1 990 in reaction to the 
program’s large waiting list and provides activities three times per month to a 
subset of youth. In its present form the program can accommodate 150 of the 
approximately 300 children on the program’s waiting list. All of the activities 
are sponsored by local businesses and community groups. 

The agency’s services and clientele were affected by the devastation caused by 

Hurricane Andrew. Approximately one-fourth of their existing matches were 

disrupted as a result of the storm because one or the other partner was displaced 

from their home. In addition a number of satellite offices were damaged or destroyed. 

The community is in the midst of rebuilding and Big Brothers/ Big Sisters has become 

involved in new programs to facilitate this process through Project Share, which 

provides families affected by the hurricane with support, advice, relief, and 

enrichment by matching them with families who were unaffected. 



Current Clientele/Users 

Big Brothers/ Big Sisters of Greater Miami provides services to Dade County 
youth between the ages of 6 and 18. Youth between the ages of 6 and 16 may receive 
matches, which last until the participants reach the age of 18. About half of the 
youth are 10- to 15-year-olds. All come from single parent families and have little 
contact with the absent parent, with the exception of some participants in the Special 
Needs program where living in a single parent household is not a requirement. The 
program’s participants have diverse economic and cultural backgrounds. 
Approximately fifty percent of the youth participating in active matches during the 
1991-1992 program year were African-American, 25 percent were Hispanic, and 25 
percent were white. 

The overall goal of the program is to provide matches to one youth from each of 
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1200 families— thereby reaching one percent of the county’s single parent families. 

Big Brothers/ Big Sisters had approximately 400 active matches during the 1991-1992 
program year and approximately 300 children on the waiting list. The majority of the 
youth participate in the core match program while most of the specialized programs 
serve only a handful of matches. A subset of the children on the waiting list 
participate in activities sponsored by Project C.A.RE.S. 

Clients are commonly referred to the program by a parent. Other referral 
sources include the youths themselves, school counselors, courts, program 
participants, and outside agencies. The program is well-known within the community 
and potential clients often learn about the program through word-of-mouth. Big 
Brothers/ Big Sisters also conducts an extensive advertising and marketing campaign 
to recruit participants and volunteers. Potential participants go through a structured 
application process and both the child and parent must indicate their consent before 
the program accepts a youth. 

Type and Makeup of SI Network 

Big Brothers/ Big Sisters collaborates with other community agencies both 
formally and informally. The core program has informal linkages with a variety of 
agencies including Dade County Youth and Family Development, school counselors, 
and the Boys/Girls Club. These agencies provide referrals and, in the case of the 
Boys/Girls Club, the use of their facilities as a meeting place for interested matches. 
Social workers assigned to the specialized match programs rely upon a different set of 
informal linkages to obtain referrals or program-related services. For instance, Teen 
Connections deals with family planning issues and has informal linkages with 
Planned Parenthood and medical clinics that may serve Teen Connections clients or 
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meet with matches. The Special Needs program has formed linkages with HRS 
developmental services, a local mental health center, and various medical service 
providers to make referrals for clients and their families. 

The Juvenile Justice program has forged informal linkages with local 
universities and minority fraternities and sororities in order to find volunteers, tutor 
advocates, and mentor advocates to participate in the program. Project CARE.S. 
actively pursues community agencies and businesses to sponsor activities to engage 
youth who are waiting for a match. In the past, they have received support and 
sponsorship from private companies, such as IBM; churches; sports teams; retail 
stores; and private clubs. 

Big Brothers/ Big Sisters has been involved in several joint ventures with 
community groups to obtain funding and create some of the program’s newest 
components. Formal linkages exist between Big Brothers /Big Sisters and these 
community groups to run the programs for which they receive funding. Spectilcally, 
Big Brothers/ Big Sisters collaborated with Switchboard of Miami to create and run its 
Teen Connections component, with TROY (Teaching and Rehabilitating Our Youth) to 
create and run the Juvenile Justice Program, and with Parent-to-Parent of Miami to 
develop and run the Special Needs Program. 

The program’s success relies on the participation of hundreds of volunteers and 
much of the agency’s efforts goes toward program marketing and volunteer 
recruitment. The agency has forged linkages with local media representatives and has 
received in-kind contributions in the form of videotapes, public service 
announcements, and segments on television and radio shows. 
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Funding Sources 

Big Brothers/ Big Sisters of Greater Miami receives the majority of its funding 
from the United Way. A large portion of its revenue also comes from in-kind 
donations, mainly advertising and public service announcements. In fact, in-kind 
contributions for advertising more than tripled from 1991 to 1992. Other funding 
sources include: 

• Special events such as the annual Toast and Roast, fund raising activities 
sponsored by the Women’s Committee, Inc. for Big Brothers/Big Sisters, and 
the agency’s major annual fund-raising event— Bowl for Kids’ Sake. 

• Support from foundations such as the Bassett, Cross Ridge, Dade Community, 
Dunspaugh-Dalton, Thomas J. Lipton, George B. Storer, Winn-Dixie, 

Wiseheart, and Southeast Banking Corporation foundations and the Mitsubishi 
Electric Sales America Corporation. 

• Contributions collected during the annual membership campaign. 

• Bequests, investments, and miscellaneous income. 

Evaluation 

Until three years ago, the agency’s primary data collection methods were 
limited to telephone calls by board members to a random sample of matches to assess 
satisfaction with the program, and a recordkeeping review conducted by board 
members. This system was reevaluated because key decision makers realized that the 
existing evaluation system was not sufficient for them to measure impact or have up- 
to-date knowledge of the service needs of the participants and volunteers. 

Currently, Big Brothers/Big Sisters maintains extensive documentation on 
each match/ case, match termination, and annual goal setting process. They prepare 
an annual report with data on: 

• The intake process. 

• Number of assessments. 

• Number of potential volunteers requesting information. 

• Source of volunteer inquiries. 
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• Number of matches made by match type. 

• The racial/ethnic distribution of the youth participant and volunteer. 

They also conduct an annual survey of current and closed matches to ascertain 
the participants’ (volunteers, parents, clients) perceptions of: 

• The appropriateness of the match. 

• Helpfulness of the staff. 

• Impact of the program (for the volunteer, the evaluation targeted perceived 
impact on the client’s grades, school attendance, teacher relations, self-esteem, 
peer relations, and family relations). 

• Suggestions for improving the program. 

The executive director at Big Brothers/ Big Sisters has expressed an interest in 
expanding their existing evaluation structure if funds for a longitudinal program 
evaluation were available. 



Brief History 

Chins Up, an acronym for "Children in Need of Supervision," is a private, non- 
profit agency in Colorado Springs, Colorado. Chins Up serves primarily youth aged 1 1 
to 18 who are referred from social services or the juvenile court system due to juvenile 
delinquency or other status offenses or who are victims of physical or sexual abuse or 
family violence. The program began in 1973 after the community expressed a need for 
emergency shelter care for troubled and runaway youth. The agency started a shelter 
program consisting of eight beds for troubled boys and girls. In 1978, in response to 
needs identified in the community for alternative educational services for troubled 
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youth, Chins Up added a state-certified special education program for the shelter 
children was added. One year later, in 1979, Chins Up began a Family Therapy 
Program that focuses on reuniting troubled youth with their families. In 1984, in 
response to the need for additional foster homes with a therapeutic component, Chins 
Up initiated a Therapeutic Foster Care Program that recruits, trains, supports, and 
provides therapeutic services to 15 foster homes and over 25 children. 

In 1988, in collaboration with other community leaders. Chins Up was a key 
player in the founding of the Joint Initiatives for Youth and Families of El Paso 
County (JI). Joint Initiatives brought together the directors of the county social 
services, youth services (juvenile justice), mental health, the county school district, 
health department, and local JTPA to reduce the number of out-of-county placements 
of runaway, homeless, or abused youth. As of 1992, 14 agencies were full or 
associate members of JI, and others from the community were seeking membership or 
attending monthly meetings as observers. In 1990, Chins Up began the El Paso 
County Family Preservation Program to expand local family preservation services. In 
1991, Chins Up was one of ten state-wide pilot programs that was awarded state 
funding under Senate Bill 94 to provide an alternative to detention for youthful status 
offenders through a program called Detention Services for Juveniles (DSJ). Both the 
family preservation program and DSJ were developed by JI and won by Chins Up 
through a competitive bidding process. In 1992 Chins Up provides a continuum of 
preventive and treatment services for troubled youth, including emergency shelter and 
residential care, special education, therapeutic foster care, family therapy, family 
preservation, and alternatives to juvenile detention. 
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Current Mission, Goals, Objectives 

The mission of Chins Up is: “to provide short-term residential care, therapeutic 
foster care, and treatment services for children and their families with the goal of 
supporting, preserving, and promoting the child’s welfare, safety, and family 
relationships wherever possible." Chins Up sees itself as an advocate for children and 
families and as provider of a continuum of services for troubled youth primarily from 
El Paso County including Colorado Springs. The philosophy of Chins Up is to work 
effectively and to collaborate with other agencies in the community, and to use 
creativity and innovation to integrate services in non-traditional ways. 

The mission of Joint Initiatives, as stated in a recent draft of the by-laws is: "to 
develop and maintain an integrated human services system, for children, youth, and 
families." In many ways, this site visit report is as much a description of Joint 
Initiatives as it is a description of the Chins Up program. Although Chins Up was the 
initial focus of the site visit and participated in forming Joint Initiatives, it is really the 
combination of Chins Up and Joint Initiatives that constitutes the highly evolved form 
of service integration operating in Colorado Springs. 

Service Configuration 

The core program at Chins Up is the short-term residential shelter for 
abandoned, runaway, and abused youth. This program is not a drop-in type of 
shelter and only takes youth who are referred from the county Departments of Social 
Services (DSS), Youth and Victim Services, the Colorado Springs Police Department, 
or private practitioners. Youth can stay for a maximum of three months and the 
primary goal of the program is to determine long-term care arrangements for the 
youth, either by returning the youth back to the parents or seeking alternative 
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custody or foster care. Youth in the residential program receive ongoing individual 
case assessments, medical attention and health services provided through the 
Community Health Center, individual and group counseling, and transportation for 
emergencies. Experienced therapists and managers axe on call 24 hours a day, seven 
days a week. Residential shelter staff include a Residential Director, two Case 
Directors, and 35 full- and part-time counselors. A key feature of the residential 
shelter is that it is not a secure fac^ty, so that youth can and often do run away for 
periods of time. Typically, youth have a time limit of 24 to 48 hours to return to the 
shelter, after which they are considered AWOL and subject to sanctions upon return. 
As long as the youth return (which most do), Chins Up does not close their case file 
during these short runaway episodes. Another important feature of the residential 
shelter is its strictly enforced rule against any form of physical contact between the 
youth and staff or among the youth. This rule was enacted several years ago in 
response to concerns that shelter residents often may be both victims and 
perpetrators of physical and sexual abuse. It remains in force despite some 
opposition from the youth (one reason why they run away for short periods is to be 
physical with each other). 

Approximately 30 percent of the youth in the residential shelter also receive 
problem-focused family therapy. In particular, youth who appear most likely to 
return to their family at the end of their shelter stay receive family therapy as long as 
their parents agree to participate. Therapy continues on an outpatient basis for six 
months after the youth returns to the family. The family therapy program is staffed 
by a clinical director, two full-time family therapists, and six part-time contract 
therapists. In addition, all shelter youth receive special educational services through 
a Chins Up-based alternative school licensed by the Colorado Department of 








138 



Education. The education program features certified special education, GED courses, 
and individualized instruction. Staff include one Education Director, two full-time 
teachers, and two teaching assistants. Finally, all shelter youth participate in a set of 
recreational and leisure activities designed to build self-esteem and to foster personal 
growth. These include educational tours, cultural events, physical activities and 
sports, guest lectures, and volunteer opportunities. 

The Therapeutic Foster Care program provides long-term therapeutic foster 
care for children ages 0 through 18 who cannot stay in their own homes for a variety 
of reasons. DSS contracts with Chins Up to recruit, train, license, and monitor 15 
therapeutic foster homes. DSS usually refers youth from a short-term residential 
placement following a court-ordered removal from the home due to physical or sexual 
abuse. Some of the youth may be referred from the Chins Up residential shelter. 
Youth stay in a therapeutic foster home until they reach 18 years of age, and the 
average age of entry is approximately 12 years. Chins Up provides therapeutic and 
supportive services to the adoptive or foster parents as well as respite care and 
recreational activities. 

The El Paso County Family Preservation Program is a recent addition to Chins 
Up and represents the preventive end of the continuum of services offered by this 
agency. Through intensive in-home intervention on a short-term basis, the Family 
Preservation Program strives to prevent unnecessary out-of-home placement of 
children and youth. This is an intensive, time-limited (4 to 6 weeks maximum) 
intervention that makes a family specialist available to the family 24 hours a day, 
seven days a week. This program closely follows the Homebuilders Model currently 
operating in family preservation programs across the country. The family specialist 
works with the family in their own home using a goal-oriented strategy with limited 
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objectives and a high level of collaboration with community agencies and institutions, 
including DSS, the Department of Youth Services (DYS), schools, food banks, and 
county welfare and mental health agencies. Referrals of families to this program come 
from the county DSS, DYS, and local school districts. The Family Preservation 
Program staff include a Program Director, three family specialists, and an intern. 

Each family specialist handles only two family cases are assigned at any one time, 
permitting intensive interaction and attention over the four- to six-week period. 

Finally, Chins Up operates the newly-developed Detention Services for 
Juveniles program out of the Zebulon Pike Juvenile Detention Center. The program 
was initiated through Senate Bill 94 in order to reduce overcrowding at Zebulon Pike 
by finding alternatives to lockup for first- time and status juvenile offenders. Youth 
arrested for a status or criminal charge and brought to Zebulon Pike at the pre- 
adjudicated stage are eligible for the program. While waiting to be locked up prior to 
their trial hearing, these youth undergo an assessment by a Chins Up staff person 
working at the detention center. Based on the risk assessment, staff assign to each 
youth a specific level of risk and may recommend alternative release if the youth does 
not appear likely to be a risk to self and/or c immunity. Alternative release strategies 
include any combination of: electronic or passive monitoring, home confinement, daily 
telephone or face-to-face contacts, random electronic voice verification through 
telephone calls to the home, curfew, or release on a personal recognizance (P.R.) bond. 
Workout, Ltd., a not-for-profit organization, provides electronic monitoring and 
tracking and collaborates with Chins Up staff. 



Current Clientele/Users 

The primary target group for most Chins Up programs are youth (and their 
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families) about to be placed in out-of-home settings or already institutionally involved 
(e.g., in detention, emergency shelter, or foster care). Such families demonstrate one 
or more of the conditions that trigger intervention by the Department of Social 
Services or the Department of Youth Services, including juvenile crime, physical or 
sexual abuse, family violence, abandoned or runaway youth, families in crisis, or 
youth with behavioral problems. Relatively little distinction exists between the youth 
and families involved in each of the program components. The specific type of 
services offered to families and youth by Chins Up depends on the referral source as 
well as on a comprehensive needs assessment conducted by Chins Up. Many youth 
enter Chins Up through the short-term residential shelter and later receive other 
services. Other youth and families first come to the attention of Chins Up through the 
Family Preservation Program or Detention Services for Juveniles. Sometimes youth 
initially involved in the Family Preservation Program may later appear at the Zebulon 
Pike Detention Center due to criminal involvement and will then be assessed by the 
Chins Up workers in this program. 

Youth in the residential shelter program are mainly white and between 13 and 
16 years of age. The El Paso County DSS typically refers youth to the program for the 
following reasons: parent/child conflict, in need of supervision, physical or sexual 
abuse, awaiting placement in a foster home, or a runaway from their own home or a 
foster home. The shelter population varies between 2-3 males for every female down 
to an approximately even sex ratio. Most of the youth were either living at home or 
were at the Zebulon Pike Detention Center before coming to the residential shelter. 
Typically, youth in the residential shelter either return to their homes, transfer to 
another residential program, or run away from Chins Up (after which the case is 



closed). 



Children in the Family Preservation Program range in age from birth to 18 
years; 36 percent of all children are 10 to 15 years old. Approximately 60 percent of 
the children classified as the identified patient are male and 40 percent are female. 
Most (61 percent) are white with 23 percent African-American and 12 percent 
Hispanic. Forty-five percent of the families are headed by a single mother while 22 
percent of the children lived with both natural parents at the time of the referral. 

Most families who participate in the Family Preservation program have annual 
incomes under $20,000, with 46 percent of all families earning under $10,000. These 
statistics are roughly similar for all other Chins Up programs, although the proportion 
of youth from minority groups may be somewhat higher in the DSJ program, as might 
be expected in a criminal justice-related program. 

Type and Makeup of SI Network 

Chins Up collaborates with an extensive, varied, and well- organized network of 
agencies. Some of these agencies are partners in the programs it operates, while 
others become involved with individual cases through monthly multi-agency team 
meetings. Finally, Chins Up collaborates with other agencies in the operation of Joint 
Initiatives. 

Chins Up works collaboratively with a variety of agencies in conducting its 
programs. Chins Up staff work off-site, at the Zebulon Pike Detention Center, to 
operate the DSJ program. At this location, they cooperate with the courts, sheriffs 
office, the detention center, and the private security company that conducts the 
monitoring of released juveniles, Workout, Ltd. In addition, Chins Up staff who 
provide Family Preservation services work in the homes of the families as well as in 
the community at large and their offices are located in a space separate from the 
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Chins Up central facility. The Family Preservation Program workers at Chins Up 
spend considerable time interacting with other agencies to meet the immediate needs 
of client families, including financial assistance, housing, emergency food, and school 
problems. 

Residential center youth receive medical services through the El Pomar Health 
Center several times a week. The Center sends a nurse practitioner to the shelter 
youth to conduct routine physical examinations, sexually transmitted disease 
screening, birth control counseling, and prenatal care. Private practitioners who 
accept Medicaid conduct eye and dental exams at their offices. Chins Up also has an 
agreement with the local YMCA to provide the youth with a "ropes" course-an 
Outward Bound type of program designed to build self esteem and teamwork. Chins 
Up also has an ongoing contract with the Pikes Peak Mental Health Center to allow up 
to three youth to be seen by two licensed therapists once a week. Through another 
agreement with the local JTPA program, youth staying in the residential shelter can 
engage in part-time work or a pre-employment training experience through the 
summer jobs program. Other special activities for residential shelter youth occur in 
collaboration with agencies in the community, such as workshops on sexually 
transmitted disease given by the county Public Health department, or a "scared 
straight" type of program provided at the Canon City jail for Chins Up youth. 

In addition to these formal arrangements, staff from other agencies involveo in 
a particular case attend a variety of staffing meetings with Chins Up staff. Youth at 
the shelter who are disruptive and violent are asked to leave. But their cases remain 
open and a monthly meeting is held to discuss youth who have been disruptive with 
all agencies that serve these youth. The meeting helps to monitor their behavior 
outside of the shelter. In addition, a Multi-Agency Review Team meets once a week to 



discuss a specific case and arrive at alternatives to a potential out-of-county foster 
care placement. The Chins Up Director of Case Management convenes the meetings 
with all agencies involved with the youth and family including JTPA, the courts, the 
youth’s attorney, the DSS caseworker, the DSS supervisor, and some treatment 
centers such as Cheyenne-Mesa, a long-term residential facility, or an emergency 
shelter, Dale House. 

A unique feature of SI efforts in Colorado Springs and El Paso County is Joint 
Initiatives. Joint Initiatives is a collaborative group consisting of the Executive 
Directors or senior administrative officials of the key social services, education, 
justice, health, and protective services agencies in El Paso County. Government 
agencies mandated to provide services for children and families are full partners in JI; 
private youth-serving agencies such as Chins Up may join as associate partners. 

JI has eight full and six associate partners. The full partners are DSS, DYS, 
the county school district, the community mental health center, the juvenile court, the 
District Attorney, the El Paso County Job Training Partnership Act (JTPA) program, 
and the county Health Department. The associate partners include: Chins Up, 
Goodwill Industries, Head Start of El Paso County, the Myron Stratton Center, and 
social services agencies from two adjacent counties. All agencies are represented on 
JI by their directors or other senior management. The full partners contribute 
$10,000 yearly and receive four votes; the associate partners contribute $2,500 yearly 
and receive one vote. Rarely does a multi-agency collaborative effort require more real 
commitment from the agencies involved, either in monetary or personnel terms. 1989 
saw the first set of by-laws developed by JI members; in 1992 they are working on 
their second by-law revision. They have yet to incorporate, and probably will do so 
only if they can get extensive waivers from state agencies enabling them to pool 
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agency resources to allow services to follow the child. They are seriously committed to 
multi-agency action and improving the efficiency and effectiveness of services to 
children and families in the county. 

Joint Initiatives grew out of the Department of Social Services’ (DSS) concern 
about the high number of youth placed in out-of-county foster homes. The costs to 
DSS and the local school district of these placements were considerably higher than 
within-county settings, due to the short-term costs of court-ordered treatment plans 
such as special education and the greater difficulty and cost of finding a long-term 
foster care solution. During the first year of operation, Joint Initiatives set as its goal 
a ten-percent reduction in the number of out-of-county placements. It achieved an 
actual reduction of thirty percent. This reduction in out-of-county placements has 
improved in the past several years and is now stable at forty-five percent. Joint 
Initiatives has strongly influenced the development of needed services in El Paso 
County by determining needed services, writing proposals to secure funding, 
requesting bids for developing a program from local agencies once funding is secured, 
and monitoring the grants and program development once a contract is let. Chins 
Up’s Family Preservation Program is one result. JI was instrumental in convincing 
the state legislature to appropriate funds (Senate Bill 94) to reduce the number of 
status offenders in institutions. Jl then applied for and received a grant under SB 94, 
and Chins Up won the contract for their newest service, Detention Services for 
Juveniles (DSJ). 

Funding Sources 

According to its audited financial report for fiscal year 1991, the bulk of Chins 
Up funding comes from fees for services paid by the Department of Social Services or 
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the Department of Youth Services; these account for 46 percent of total program 
support. Other sources of funds are as follows: 19 percent from foster care fees, 1 1 
percent from after-care services funded by a variety of state agencies, 8 percent from 
the DSS family preservation budget, and 8 percent from the Department of Education 
for its alternative school. Smaller amounts of support come from United Way (3 
percent), child nutrition, family therapy fees, the Colorado Trust, and independent 
contributions. Given the program’s commitment to providing services to youth under 
public agency supervision it is not surprising that the bulk of its support comes from 
fees for services that are either court-ordered or paid for by the county DSS office. 

Evaluation 

Chins Up is highly committed to monitoring its services and client 
characteristics using a relatively comprehensive computerized information system. It 
is also highly motivated to conduct more evaluation research and would like to do 
some longitudinal follow-up of the youth in the residential shelter and family 
preservation programs. However, staff and management have little or no evaluation 
expertise, and both time and resources to do these kinds of activities are limited. 

Chins Up has not had extensive experience with evaluation research to date. 
The only program currently being evaluated is the Detention Services for Juveniles, as 
part of a state-wide evaluation of the Senate Bill 94 programs. The evaluation is being 
conducted by a private contractor and the study has encountered a lot of problems — 
particularly the degree to which program staff get feedback. Currently Chins Up feels 
much resistance to the state-wide evaluation because the DSJ workers perceive it as 
not meeting their needs, being overly intrusive and rigid, and not being done 
competently (especially the forms they are given to complete, which appear to lack 
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sufficient operational definitions). The contractor wanted all service and client 
information data stored at a single site in Boulder and DSJ refused. They decided to 
keep control of their own data rather than share it with the state-wide evaluator and 
Chins Up staff are currently developing their own computerized database, containing 
comprehensive information about the types of youth in the program and what 
happened to them, particularly whether they re-offend. This evaluation also suffers 
from having many different stakeholders invested in the results in different ways, so 
that for some the results might support the utility of the program whereas for others 
the results might suggest that it should be shut down. A JI oversight committee is 
attempting to mediate the expectations of all involved parties. 

Each Chins Up program component maintains extensive records on client 
characteristics, assessment of risk, family backgrounds, service participation, and 
case disposition and referral destinations. All information from the separate program 
components are entered into a computerized database that generates reports to 
summarize program participation by individual cases, hours of direct and indirect 
service provided, case file status, and dispositions. For example, they are able to 
report that the outpatient family therapy achieved a success rate of 76 percent, with 
success defined as the child remaining at home and continuing in school and/or 
working full- or part-time. They report that 90 percent of the target children in 
families served by the Family Preservation Program remained at home with their 
families, despite the high risk of out-of-home placement for the target youth at 
program entry. They also report that the Multi-Agency Review Team has successfully 
reduced the number of out-of-county foster care placements of youth by 25 percent 
compared to the state average which has shown a consistent increase during the 
same period. In addition, the information system is able to calculate the cost per unit 



of service provided to each youth in the program and identify which funds supported 
those services. This has potential utility in a cost-effectiveness component to an 
evaluation. 



GARFIELD YOUTH SERVICES 



Executive Director: 
City/State: 

Phone Number: 



Debbie Wilde 
Garfield County, CO 
( 303 ) 945-9300 



Brief History 

Garfield Youth Services (GYS) began in 1976 as an ad hoc effort of a group of 
concerned parents in Rifle. Colorado. Their children had described to them some of 
the substance-abusing and other risk-taking behaviors of youth at school, and asked 
the parents what could be done. The group incorporated as Let’s Work It Out, Inc., 
hired its first director, and aimed to provide drug information and education to youth. 
The next year saw a shift to alcohol and drug prevention work. In 1978 the 
geographical service area expanded beyond Rifle to all of western Garfield County, and 
the name of the organization was changed to Garfield West Youth Services. That year, 
the program provided prevention services to approximately 450 youth. The change to 
the present name, Garfield Youth Services, came in 1979 when the program expanded 
to serve ‘he whole county. 

Ove*- the intervening years GYS has gone from a budget of $30,000 and a staff 
of one to a budget of around half a million dollars and a staff of 15, with varying 
numbers of volunteers. In some years VISTA volunteers and interns have also 
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augmented the staff. Services have been added or expanded every year, including 
services to bring in parents of youth served and other community members, 
expansion of the age range of youth served, treatment services in addition to 
prevention, detention and placement alternatives, new varieties of prevention activities 
such as mentoring and recreation /activities programs, and host homes as temporary 
shelter. 

In some instances GYS has identified service needs and acted to fill them. It is 
more common, however, for community agencies such as the schools, courts, and 
social services to approach GYS to supply services to youth that the agencies 
themselves cannot provide with existing resources. GYS has become more selective in 
deciding what to take on and what to pass over. The board and staff of GYS routinely 
refer to their mission statement and agency purpose in deciding whether or not to 
expand in a new direction. 

Throughout its history, GYS has worked to be perceived as an agency for all 
youth, and not just for "bad kids. " It has also stressed the importance of having all 
members of the community care about and work toward improving the chances of all 
youth to have a successful life and its willingness to work with the entire community 
to this end. In a rural area characterized by distinct regional differences in 
orientation and resources, this inclusive and cooperative approach has been both 
absolutely necessary and highly effective. 

GYS director Debbie Wilde is articulate about the unique aspects of developing 
services in a rural area where before GYS there were virtually no services for youth. 
First, there were no turf battles to fight, because no other agencies already had a 
claim on a particular type of service. Second, the community recognizes all new 
services as needed and welcome. Third, the welcome new services receive depends on 
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the program’s ability to develop the new service with the full cooperation of each local 
community at every step. Wilde stresses how GYS presents and interprets each new 
program in ways that each local community will understand, including changing the 
program name slightly if that seems important to community acceptance. Wilde notes 
the importance of developing community members’ sense of responsibility for "our 
children and youth," and encouraging their participation and involvement rather than 
leaving things to some official agency. For instance, when community members 
blamed the schools for not doing enough to prevent youth behavior problems, GYS 
offered alcohol and drug prevention programs to the schools and training for the 
teachers. GYS then helped the schools respond to the community by saying "this 
[GYS program) is what we [the schools] are doing, what are you [the community! 
doing?" Wilde believes that this approach helped reverse the attitude that youth were 
someone else’s problem and began to get community people thinking about their role 
in supporting all youth. 



Current Mission, Goals, Objectives 

GYS states its mission as "providing opportunities for ALL YOUTH to be 
responsible, contributing members of society and working with their families toward 
this end. Through prevention, advocacy and direct services, Garfield Youth Services 
strives to enhance the quality of life in our communities." GYS also has specific 
written goals and objectives to make this mission statement more explicit. These 
goals and objectives are more short-term than the mission statement, and frequently 
pertain to goals and objectives for specific new undertakings. The entire history and 
development of GYS indicate that the "all youth" part of the mission statement is 
taken very seriously in the development of specific goals and objectives. A great deal 
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of the agency’s prevention work has developed from thinking about how to reach and 
serve all youth, all parents, and all families. Yet GYS does not strive to be all things 
to youth-its major focus is in alcohol and drug prevention work, treatment issues 
stemming from the alcohol and drug involvement of youth, and youth involved with 
the criminal justice system, often as a result of alcohol or drug use. GYS has decided 
not to expand in a major way to include services related to adolescent pregnancy 
(either prevention or care), reasoning that the activities of other community resources 
were already adequate to handle the need. However, GYS does provide group sessions 
on these topics in areas of the county with no prevention resources. 

Service Configuration 

GYS has an extensive range of both prevention and treatment services. 
Prevention services include school-based presentations to youth and to parents, 
presentations to community groups, and PALs (a mentoring program). The program 
offers parents and adult community members groups for stepparents, parenting the 
young child, powerful parenting, being a new parent, parent support and bridging the 
gap (for parents and adolescents together). They offer Project C.H.A.R.L.I.E., an early 
intervention program, to elementary school classes. They conduct drug and alcohol 
awareness classes, prevention classes, and refusal skills classes in middle and high 
schools. In the community, they run groups for youth on family change, self esteem, 
drug and alcohol awareness, children of alcoholics, theft/petty theft, death and loss, 
communication skills, teenage pregnancy and STDs prevention, sexuality education, 
defensive driving classes, and young men’s and young women’s groups, social skills 
and feelings groups. The PALs program involves adult and teen community members 
as mentors for more than 100 youth; GYS also schedules monthly activities for PALs 
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who have a match (a mentor) and for youth on the PALs waiting list who have not yet 
been matched. 

GYS’ system of 10 active Host Homes bridges the gap between prevention and 
treatment services. These homes serve as the community’s youth shelter, providing 
temporary emergency residence for youth who cannot or will not stay in their own 
homes until a permanent arrangement can be developed. GYS developed these homes 
when it became apparent that an occasional need arose for youth emergency shelter, 
but not enough demand to justify setting up a full-time shelter. GYS trains the host 
home families, places youth in the homes when necessary, and supervises the 
placement. 

In the treatment area, GYS offers crisis intervention counseling to youth 
referred by police departments, the courts, the district attorney, probation, the 
schools, the departments of social services and mental health, the Division of Youth 
Services of the Department of Institutions, parents, friends, and self-referral. The 
crisis team produced the most referrals from any single agency (33 percent), but 34 
percent came from all the courts combined. Counseling typically lasts 4 to 6 weeks 
and can be renewed if necessaiy. A recent addition to GYS* treatment options is case 
management through the Community Evaluation Team, which is supported by a grant 
from a new state program (Senate Bill 94) to reduce detention placements and youth 
commitments to the Department of Institutions. This team is described below, in 
relation to the service integration network in Garfield County. 

Current Clientele/Users 

GYS sees youth clients and their parents for treatment services (crisis 
intervention, case management, and restitution), youth and adults for school-based 
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prevention services, youth and adults for community-based short-term groups on 
various topics, and youth and adults in the PALs (mentoring) program. Of their new 
service clients (758 youth) for FT 1992, 59 percent were male and 41 percent were 
female; 39 percent were 16 and older, 49 percent were 10 to 15, and 12 percent were 
younger than 10. GYS also handled 239 alternative sentencing clients, 25 restitution 
clients, and ran groups for 304 participants. School-based prevention interventions 
reached almost 3000 youth (not necessarily unduplicated) and about 750 parents, 
and 115 teachers attended training sessions. Several hundred youth attended short- 
term groups located in the community. PALs made 52 new matches with many more 
youth, teens, and adults participating as ongoing Junior, Teen, and Senior PALs. 

Type and Makeup of SI Network 

GYS is part of an SI network in both a "back-end" and a "front-end" way. By 
"back-end," we refer to the typical image of SI, in which a program with a client can 
access services for that client through a formal network with other providers. The 
most straightforward instance of this for GYS is the Community Evaluation Team, a 
multi-agency team involving GYS, mental health, social services, the courts, schools, 
and other relevant agencies as necessary. The team meets regularly for three hours 
and handles six clients/ families in each meeting. Youth often attend, and parents 
attend in about 90 percent of the cases. The outcome of each meeting is a service 
plan involving two or more agencies, to which the youth, parents, and relevant 
agencies agree. Staff say this team approach cuts the time needed to arrange the 
elements of a service package from several days to half an hour. In addition, agencies 
that have committed themselves during team meetings to provide or arrange for 
certain services follow through more quickly than they did before the team began to 



function. 



But the more interesting aspect of SI in Garfield County is at the "front end." 
When government agencies in the county (schools, courts, social services, police) 
identify a service need for youth that they cannot fulfill, they turn to GYS. GYS will 
consider developing the service, and will discuss how the new service will relate to 
existing agencies, whether GYS is the right place for the service, and other relevant 
issues. Often in its history, GYS has developed the service, making itself, over the 
years, the mortar or glue that holds the system together. It is seen by both public 
agencies and private citizens as "the place for youth" in the county. GYS also 
identifies unmet needs through calls to the hotline it runs. Once a need is identified, 
a collaborative process begins between GYS and other agencies to see lv w the need 
will be met. Sometimes this has resulted in GYS developing new services; sometimes 
it has resulted in other agencies taking on the task. According to GYS’ director, in a 
rural county where there are no services to start with, practically anything is welcome 
and service development is a cooperative enterprise. We think this cooperative 
development of services needed in the community is an important aspect of service 
integration that is often overlooked in the focus on improving the process of serving 
clients already in the system. We discuss it further in Chapter 6. 

Funding Sources 

GYS is paid by some local government agencies to deliver services (e.g., by the 
schools to do some prevention workshops and by DSS to handle early intervention 
with first- time referrals), but almost half of its funding comes from state and Federal 
contracts to provide services that local agencies cannot offer with their own resources. 
Of its FY 1992- 1993 projected budget, GYS received support from the following 
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sources: 



State contracts 19% 

Federal contracts 27% 

Garfield County Government agencies 10% 
Other counties 4% 

Garfield County Schools 2% 

Drug Free Schools 8% 

In-kind Rent 4% 

Foundations 7% 

United Way 6% 

Contributions 5% 

Operating Revenue 7% 

Other <1% 

for the following services: 

Drug and Alcohol Prevention 24% 

Drug and Alcohol Intervention 6% 

Diversion 8% 

Drug and Alcohol Offenders 1 1% 

Victim Services 7% 

Runaway Youth 1 4% 

Case Management 2 1% 

Management/General 4% 

Fundraising 5% 



Evaluation 

Program staff do not see GYS as able to conduct an outcome evaluation on 
their own at this time. But they would welcome assistance in conducting evaluations 
of both their treatment and prevention activities if the evaluation design reflects the 
scope of their program activities and impacts on clients and community. GYS staff 
are currently involved in the following data- related activities that could form part of an 
evaluation framework: 



• Program staff produce computerized program statistics on: client age, sex, 
residence, ethnicity, number served and reason for referral in each type of 
treatment service: and number of presentations, number of attendees, and 
location for each type of prevention activity. 

• Crisis intervention clients fill out an assessment survey at intake on behaviors, 
attitudes and feelings in the areas of family, abuse, drug/ alcohol, self esteem, 
mental health, behavior, life skills, peers, perception of future, school, and 




155 




community. Staff use this assessment to identify issues to explore in 
counseling. Clients complete the same assessment at termination, and staff 
compare the composite scores derived from each administration. The 
expectation is that the scores will decrease significantly. If they do not, or if 
they go up, staff offer additional counseling. 

• Participants evaluate all prevention activities. In school presentations, both 
students and teacher complete an evaluation form. In parenting and other 
community presentations, the participants complete an evaluation. In both 
instances the evaluations serve as the feedback about the session and how it 
went; only a few of these activities use pre-post testing to assess changes in 
knowledge or attitudes. 



I HAVE A FUTURE 

Deputy Director: 

City/State: 

Phone Number: 

Brief History 

The I Have a Future program began officially in 1987 as an adolescent 
pregnancy prevention program, although its original grant application stated broader 
objectives including anti-violence, vocational preparation, alcohol and drug abuse 
prevention, and academic achievement. Its original approach used case management 
and brokering of services, including brokering for some enrichment activities (e.g., 
karate, dance). Staff of various community agencies came to the community center in 
the housing projects where the program operates to deliver these services, while 
program staff provide primarily case management. The community center location 
was shared with many other pn grams, and could not offer I Have a Future a space it 
could call its own and where the youth could feel a sense of ownership. Further, it 
could not provide space for the health clinic part of the program. As a last problem 



Lorraine Williams Greene 
Nashville. TN 
( 615 ) 327-6100 
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with this arrangement, the program did not control the contents of the services offered 
through these brokered arrangements, and could not expect to integrate the value 
system it tries to convey to its users into all the services offered. 

When the present Deputy Director, Dr. Lorraine Greene, joined the "Future" 
staff in February 1989, she helped to change the program structure and emphasis 
toward an approach more culturally sensitive to the situation of African-American 
youth, one that incorporates a clear value system into every element of the program, 
and one that explicitly addresses the broad array of problems and prevention needs 
confronting youth. It was clear that to do this, program staff would have to be able to 
do more than case management; they would need the skills to run groups, convey 
values and principles, and actually provide many of the services that had previously 
been available through other agencies. At the same time the Nashville-Davidson 
Housing Authority committed one housing unit in each housing project to be used as 
program space for I Have a Future. The new approach thus combines case 
management with curriculum modules and activities for youth and parents taking 
place at sites completely under the control of the program (one large apartment in 

t 

each of two public housing projects). Carnegie Corporation offered I Have a Future a 
technical assistance team to help develop the content of the different modules, which 
include family life education, pre-employment, pro-social behavior, conflict resolution, 
and alcohol and drug abuse prevention. Each module teaches skills and then gives 
youth opportunities to practice the skills in different settings. Each also teaches 
youth how to think about and apply the Nguzo Saba Seven Principles of Blackness 
(unity, collective work and responsibility, purpose, self-determination, cooperative 
economics, creativity, and faith) in daily life situations. The program still accesses 
some services in the community, such as karate and dance classes. During previous 
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years other community agencies and organizations provided some parts of the Parent 
Empowerment Program. 



Current Mission, Goals, Objectives 

I Have a Future has a mission: 'To address the problems confronting poor 
African-American youth through a comprehensive effort of prevention, addressed 
toward early pregnancy and childbearing, substance abuse, violence, and school 
failure." This mission is elaborated in goals related both to client outcomes and to 
program development. The program states as its goals that it intends: 'To develop a 
replicable community-based, life-enhancement program that promotes a significant 
reduction in the incidence of early pregnancy and childbearing and other harmful 
behaviors among high-risk male and female adolescents between the ages of 10 and 
17 ." 

The goals of I Have a Future have been extended to five specific objectives: 

• To improve knowledge, attitudes, and behaviors related to personal health, 
human sexuality, drug and alcohol abuse, homicide and violence reduction and 
other factors which may place adolescents at risk. 

t 

• To provide greater access to, and increase utilization of, comprehensive 
adolescent health services and social services, including contraceptive 
availability. 

• To improve socially adaptive/appropriate behaviors with particular focus on 
school achievement, pre-vocational skill development, and delinquency rates. 

• To enhance the ability of high-risk adolescents to overcome environmental 
barriers in attaining tht skills necessary to pursue meaningful employment and 
educational opportunities with the promise of upward mobility. 

• To engender a more positive self-concept and constructive attitude toward 
community, family life, and the future through the use of the Nguzo Saba 
Principles. 

I Have a Future has the additional program objective of involving adults members of 
its two housing project communities in activities that will support them and their 



children in resisting drug and alcohol dependency and taking greater control of their 
lives and circumstances. 

Service Configuration 

I Have a Future organizes its services around curriculum modules delivered to 
small groups of youth, coupled with a thorough assessment, case management, and 
tracking system. In addition, the program offers primary health care on-site. 

Everyone is required to participate in three of the modules: pro-social behavior, 
family life education, and CHARM (for girls) or MATURE (for boys). Pro-social 
behavior covers such topics as how to behave in groups, decision- making and 
problem-solving skills with a particular emphasis on alcohol and drug abuse 
prevention, and respecting oneself and others. Youth must complete the pro-social 
behavior module before they can participate in other modules or activities. Family life 
education addresses the stereotypes and realities of family life and covers issues 
related to adolescent sexuality and prevention of too-early pregnancy and 
childbearing. CHARM and MATURE are on-going modules for girls and boys, 
respectively. They address issues of grooming, dress, hygiene, and self-respecting 
behavior. These modules give boys and girls a chance to discuss things in same-sex 
groups that they might feel less comfortable discussing with the opposite sex present. 
Youth in the program may attend CHARM and MATURE at any time. 

After completing the initial required modules, youth may choose from among a 
variety of other modules and activities, including tutoring, self-defense, computer 
skills, pre-employment, creative movement/dance, sports, art classes, outings, conflict 
resolution training and violence prevention, peer counseling, and entrepreneurship. 
The program is meant to accommodate youth staying as long as 7 years (coming in at 
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10 and staying until they graduate from high school), so there is always something 
new or different to do. 

Case management begins with a thorough assessment within two weeks of a 
youth’s program entry. Counselor and youth then discuss needs and preferences and 
how these can be met. Once finished with the initial required modules, youth may 
select activities or modules that appeal to them or the counselor may recommend 
certain activities based on his or her assessment of the youth’s circumstances. Any 
module may be repeated, and several modules are designed to be on-going, with youth 
attending for as long and as often as they like (CHARM. MATURE, conflict resolution). 
Youth achievement within each module is assessed by pre- and post-testing using 
paper-and- pencil instruments. Every month, the counselor and youth meet to see 
how things are going; progress notes are written on every youth every month. Every 
six months there is a major reassessment of each youth in terms of achievement of 
past goals and setting of new ones. 

Other activities involve opportunities for service. Youth may be selected as peer 
counselors, which are paid 10-15 hour-a-week positions that give youth responsibility 
for monitoring program activities, giving speeches and presentations in the 
community, helping younger children with schoolwork, overseeing the latchkey 
program for 6- 10-year-olds, recording everyone’s grades on school report card days, 
and similar duties. Youth who are not officially peer counselors may (and do) help 
others with schoolwork and offer other supports as appropriate. Most youth are in 
the entrepreneurs club, in which they learn business skills, operate a business of 
their own, and earn money. 

Services to youth are complemented by programs for adult residents of the two 
housing projects where 1 Have a Future operates. The Parent Empowerment Program 
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offers both 4-week and 12-week seminars for parents on issues of co-dependency and 
alcohol and drug abuse, such as self esteem enhancement, dealing with depression, 
and other related matters. Some graduates of these programs receive additional 
training to become recruiters, peer counselors, and supports for first-time adult 
participants. 



Current Clientele/Users 

I Have a Future provides services to 10- 17-year-olds who live in either of two 
public housing projects in Nashville. Participants are spread relatively evenly over the 
age range. Participants are 98 percent African-American; 51 percent are male. At any 
time, about 150 youth actively participate in services. More than 500 youth have 
come through the program since services began in 1988. 

Most youth refer themselves to 1 Have a Future. They learn about the program 
through word of mouth, a friend or sibling in the program, or presentations made by 
the program in schools and community groups. Referrals also come from concerned 
parents and counselors and social workers in schools. A few referrals come from 
juvenile court or probation, which send youth to participate in I Have a Future’s 
conflict resolution module. Those who live in the projects sometimes stay on after 
their obligatory participation ends, but those from elsewhere have a hard time getting 
to the program because transportation is only available while they are fulfilling their 
court obligation. 

Every new participant signs a contract upon entering I Have a Future. The 
contract commits the youth to have a physical examination within 60 days (available 
free at the program site), participate in the 8-week module on pro-social behavior, 
participate in either the CHARM (for girls) or MATURE (for boys) module, and 
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participate in the 14-week family life education module. Participation in at least one 
of the modules must happen within the first 60 days; youth must complete the pro- 
social module before attending any other module. Each youth discusses further ways 
to participate in I Have a Future once he or she has successfully completed the 60- 
day commitment. 

Other I Have a Future participants are upwards of 250 adults from the two 
public housing projects (not necessarily parents of "Future" youth) who have 
participated in 4-week Parent Empowerment seminars. About half of these parents 
have gone on to participate in the 12 -week extended Parent Empowerment seminar 
offered by I Have a Future, or other chemical dependency or co- dependency support 
resources such as AA, NA, Al-Anon, ACOA, or individual substance abuse counseling. 
Some have become involved in local tenant councils, educational activities, developing 
day care resources within the housing projects, and other activities on their own 
behalf or on behalf of their families and communities. About 30 parents who have 
been through Parent Empowerment serve as recruiters, counselors, and trainers for 
this part of the program. 

Type and Makeup of SI Network 

I Have a Future provides a summer JTPA program with Federal funds on-site 
through a contract with the City of Nashville. Tennessee State University provides 
educational enrichment workshops and tutoring on-site to I Have a Future 
participants through another contract. Other interagency agreements with 
community agencies are for short-term resource sharing. For instance, for several 
years a local Catholic Church operated parenting skills workshops that were part of 
the I Have a Future Parent Empowerment Program. I Have a Future is a member of 
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the Alliance for African-American Males, a consortium of community agencies. The 
Alliance occasionally refers youth to I Have a Future. I Have a Future may also have 
occasion to call on the services of other programs in the Alliance when participants 
need them. 

The program would have more collaborative activities if recent fundraising 
efforts had succeeded. I Have a Future and several of the schools attended by 
program clients have written joint grant applications to support Future programming 
at school sites. Future staff now go to the local high school at least monthly to do 
special activities and also run some of the program’s modules in the schools. Most of 
the participants are Future youth, but others may also participate. I Have a Future 
has enough of a presence in this school to be listed as a club in the school yearbook. 
There are plans for 1 Have a Future to develop and staff a health clinic in the school. 
The school has made space available; proposals to raise money to staff the clinic have 
not been funded, but Future and the school will keep trying. If it opens, this clinic 
will be available to all students, not just to Future participants. 

In general, I Have a Future uses a variety of community resources and obtains 
referrals from a number of agencies. It also succeeds in providing a comprehensive 
program for at-risk youth geared to prevention and to life options and empowerment. 

If some recent fundraising activities had been successful, it would be involved in some 
more collaborative arrangements with several schools. But as things stand, I Have a 
Future does not do much in a service integration framework. 

Funding Sourcr.t 

I Have a Future receives 90 percent of its funding from private sources: the 
Carnegie Corporation, Bill and Camille Cosby (as individuals), the William and Flora 



Hewlett Foundation (for the entrepreneurs program), and the William T. Grant 
Foundation (for evaluation). About 10 percent of program support comes from the 
State of Tennessee Department of Mental Health and Mental Retardation to fund the 
Parent Empowerment Program and the latchkey program. In addition, program space 
is donated by the Nashville-Davidson Housing Authority. 

The program began as a demonstration with major support from the Carnegie 
Corporation. It was never funded up to the level of its original design, and has been 
operating with many of its staff at half-time rather than at the full-time level originally 
planned. The program has recently been renewed by the Carnegie Corporation for 
another two years, and by the Hewlett Foundation for two years. The State of 
Tennessee support is small but stable. But the program is not sure where it will get 
the remaining part of its budget (approximately 40 percent) if the Cosby funding is not 
renewed. Program staff have written several unsuccessful grant proposals and are 
looking for additional sources of support. 

Evaluation 

Because it was established as a demonstration, I Have a Future has been 
involved in evaluation and documentation of its activities since it began. Conducting 
these evaluation activities is part of the program’s obligation under its demonstration 
funding; it receives financial support from the William T. Grant Foundation 
specifically for evaluation. 

The evaluation design was developed by the staff of the Meharry Medical 
College (which houses the program) as part of the initial grant application whose 
funding started the program. The design remained essentially as written, buf; 
modifications were made to accommodate suggestions made by reviewers during the 
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grant review process. These same staff developed the initial instrumentation. When 
the program shifted emphasis after Dr. Greene arrived, the new staff used the original 
instrumentation as a base and added components that assessed newly important 
program aspects (e.g., values orientation, issues related to sexuality, self-concept 
measures, and measures to assess the effects of many specific components of the 
program’s curriculum modules). Staff made inquiries of knowledgeable people in 
assembling their current instrumentation, but basically designed and developed their 
evaluation system themselves. 

The first data collection effort was a community needs assessment that 
established the parameters of the program. The evaluation of I Have a Future called 
for in their demonstration plan used a quasi-experimental pre-post design, comparing 
teens in two North Nashville housing projects where the program operates to teens in 
two East Nashville housing projects that do not have the program. As part of this 
evaluation, three annual waves of individual interviews with sampled teens were 
conducting starting early in the program’s history and continuing through 1991. The 
results based on these surveys have not yet been published; according to the Deputy 
Director, preliminary results indicate that the program has had a very positive impact 
in reducing teen pregnancies and helping youth avoid participation in other problem 
behaviors that are part of the program’s prevention effort. In addition, for each new 
participant staff administer a thorough assessment and record the results. Staff 
document monthly progress in notes, and conduct a bi-annual review, reassessment, 
and update of each youth’s service plan. Finally, every youth participating in a 
module completes an assessment before and after participation, to document learning 
and attitude change. Records of participation document program impacts on the 
adults participating in the Parent Empowerment seminars. 
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OASIS CENTER 



Executive Director: 
City/State: 
Telephone Number: 



Mazy Jane Dewey 
Nashville, TN 
(61S) 327-4455 



Brief History 

Oasis Center was originally conceived as a drop-in center to provide counseling 
and crisis intervention for clients of all ages. The "Rap House" opened in 1969 in 
response to concerns about drug use in the community and the incarceration of 
juvenile status offenders in adult jans. A health clinic was added the following year 
and a crisis shelter, Oasis House, opened in 1976. From the onset, the Center has 
focused its programs and activities on prevention and treatment (e.g., school-based 
drug and alcohol abuse prevention education and crisis counseling). 

In the early 1980s, the Oasis Center’s staff and Board of Directors refocused 
the Center’s efforts from serving individuals of all ages to providing comprehensive 
services to meet the needs of teenagers and their families. By focusing on teenagers, 
the Center could proactively serve adolescents at the point at which they are mos t 
likely to get "off track." 

Oasis Center’s previous executive director, Della Hughes, was well-informed 
about regional and national needs with regard to youth services and related issues 
and helped to focus the Center’s activities in its formative years (1979-1988). She was 
also instrumental in developing a strategic plan to provide comprehensive services to 
teenagers that involved changes in the composition of the Board of Directors and 
expanded fundraising capabilities. The Oasis Center modified the role and 
membership of its Board of Directors from a loose network of social service providers 



to a board composed of community members committed to the Center’s mission and 
with the influence necessaiy to promote fundraising opportunities within the 
community. The current Board includes representatives from local businesses, 
educational institutions, community volunteers, and a high school student 
representative. 



Current Mission, Goals, Objectives 

Nashville’s Oasis Center is a private, non-profit, community-based agency 
providing a comprehensive set of crisis services to teens and their families. The 
Center’s mission is to empower youth and their families to meet the demands of 
adolescence, primarily through the provision of youth-centered services. Its primary 
goal is "to provide comprehensive services to help teens and their families succeed." 

The Center’s objectives include: 

• Providing teens with help for immediate problems. 

• Helping teens to resolve their underlying problems. 

• Facilitating the teen’s transition from adolescence to adulthood and preparing 

teens for the responsibilities of adulthood. 

In 1988, Oasis Center developed a five-year plan that identified the following 
administrative and service-related long-range goals: 

• Identifying and filling service gaps. 

• Involving and serving minorities. 

• Advocating for youth and family service needs at all levels. 

• Continuing to use sound agency management. 

• Diversifying the program’s funding base. 

• Obtaining permanent facilities. 

• Refining financial and data management systems. 
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The Oasis Center periodically reassesses its goals and objectives and the 
services it provides in response to identified client and community needs. For 
instance, after an internal review revealed that the Oasis Center’s foster care services 
were not meeting the needs of the teenagers being placed, the Center phased out the 
program and re-directed its energies toward home-based and independent-living 
services. 

Service Configuration 

Many of Oasis Center’s programs have evolved from needs identified during the 
course of service delivery. The Center currently offers a range of residential, 
educational, and vocational services, including: an emergency shelter, community- 
based counseling (early intervention for drug and alcohol abuse prevention), family 
preservation and home-based services, sex abuse prevention, community outreach 
activities, youth opportunities programs, and life transitions programs. Clients may 
enter the service delivery system through any of these programs or services. All 
clients receive a detailed intake interview and assessment at which time staff identify 
their service needs and develop an action plan. The intake and referral interview 
records referral source, presenting problems, and related information: information 
about the client’s family and living situation; service history; and service plan. The 
Oasis Center’s Assessment form gathers detailed information about the client’s gender 
and ethnic group, family, education, legal status, social and peer-related activities, 
general health, emotional and psychological state, and history of drug use. 

Residential Services. The Center provides a short-term emergency residential 
shelter for youth aged 13-17, and a residential independent living program for youth 
aged 17 to 21, While residents learn about the emergency shelter from a variety of 



sources (e.g., school, juvenile court, counseling agency) the most common form of 
entry into the shelter is through the Safe Place outreach program organized by Oasis. 
Residents receive temporary shelter; receive individual and group counseling, and 
family counseling when appropriate; attend school, most often within the shelter; and 
participate in recreational activities. Shelter residents have included homeless youth, 
youth from families that are in crisis, and youth in the custody of a state agency, 
among others. 

The independent living program provides a residence for older youth until they 
can establish themselves independently. It also provides non-residential services (e.g., 
employment counseling, independent living skills training). Youth enter the 
independent living program through several Oasis service components, and also 
through referrals from outside agencies. 

Community-Based Counseling. Counseling services include individual, group 
and family counseling and a variety of group programs (e.g.. Early Intervention, 

Alcohol and Drug Abuse Prevention, and Suicide Prevention programs). The Center 
offers outpatient services to families on a sliding fee scale; services range in duration 
from two months to two years. Crisis walk-in services are available to individuals in 
need of immediate assistance when the residential shelter or clinical services 
component is unavailable. 

Early Intervention and other topic-specific groups run for eight to ten weeks 
and provide participants with the opportunity to practice group skills (e.g., teamwork, 
group interactions) and deal with common issues (e.g., sexual abuse, drug or alcohol 
abuse). Oasis Center also offers group sessions for interested parents. 

Outreach Activities. Project Safe Place is a 24-hour outreach service for youth 
in crisis between the ages of 13 and 17. This national program is sponsored locally by 
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Oasis Center. The Nashville community has designated certain businesses and public 
locations as "safe places" from which trained staff help youth to contact Oasis Center 
and arrange for services or transportation home or to the residential shelter, i^ 
necessary. Many youth enter the Oasis Center service system by referring themselves 
through Safe Place. The Safe Place headquarters and hotline is located in the Shelter. 

Youth Opportunities Program. This program includes a youth employment 
component for teens aged 16 to 19, and training in life skills and career planning to 
prepare teens in the custody of the Tennessee Department of Human Services for 
independent living. 

Current Clientele/Users 

The Oasis Center provides services to 13-21 -year-olds and their families, most 
of whom reside in urban Davidson County, Tennessee. More than fifty percent of the 
youth served are between 12 and 15 years old. Approximately twenty percent of 
program participants are African-American and almost eighty percent are white. The 
demographic profile of the client population is consistent with that of Nashville. Oasis 
Center programs served approximately 3,000 youth in 1991. This figure does not 
include over 9,700 youth educated about the Safe Place program through school 
presentations and other Safe Place publicity. 

Clients nay enter Oasis Center through any of its programs and services. In 
most cases, clients refer themselves into Oasis Center programs. They learn about 
the program through word of mouth, friends who have participated in Oasis Center 
programs (especially the Early Intervention program located in schools), and Safe 
Place outreach activities. Safe Place participants refer youth to the Oasis Center for 
services or to the Center’s youth shelter for immediate care. Other referral sources 



have included counseling agencies, teachers, juvenile court, and state agencies having 
custody of a child. 

Most Oasis Center programs focus on youth but may involve the family in 
services and treatment. Upon entering the service delivery system, youth are assigned 
a case manager who matches the youth and his or her family with needed services. 
Family members may receive crisis counseling or longer term counseling services to 
deal with a range of family issues (e.g., communication, parent/child relationships, 
drug and alcohol abuse). Families who have an adolescent at risk of out-of-home 
placement receive six months of intensive in-home services. Oasis Center also serves 
parents directly by offering Parenting Skills Groups at the Center and workplace 
parenting programs. 

Type and Makeup of SI Network 

The Oasis Center facilitates a client’s access to the services of community 
agencies and resources that have forged linkages (both formal and informal) with the 
center. A downtown Nashville Health Clinic provides transportation and health care 
to residents of the Emergency Residential Shelter and the Nashville school system 
provides on-site schooling, both through formal contracts with Oasis Center. 

Most of Oasis Center’s interagency relationships are informal and have 
developed through staff participation on community boards and committees. Oasis 
Center is a member of the Adolescent Services Network, a forum composed of many 
youth-serving agencies in Nashville that meets monthly to discuss issues relating to 
chronic runaways. The Center also meets with other emergency or crisis services 
agencies in an Emergency Services Network to deal with the needs of youth who have 
been neglected or abused by alcohol or drug abusing caregivers. 
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Oasis Center receives referrals from numerous agencies and provides 
information and referral to clients needing services not provided by the Center. For 
instance, Oasis Center refers parents of substance abusers to the Alcohol and Drug 
Abuse Council for services and maintains informal linkages with a local agency for 
suicide prevention/evaluation referrals. 

Oasis Center interacts regularly with the Juvenile Court and the Department of 
Human Services, which place youth under their custody in the Center’s shelter and 
other programs. Juvenile Court and DHS informants perceive Oasis Center as 
responsive to the needs of youth and families and regularly refer youth to the Center 
for services (such as emergency shelter, home-based, counseling, independent living, 
and GED preparation services). 

The linkages between the Oasis Center and both Juvenile Court and DHS 
extend beyond referring clients to the Oasis Center for services. Oasis Center has 
helped the Juvenile Court to develop a Crisis Intervention group in its detention 
facilities. Additionally, Oasis Center makes it a practice to refer youth to DHS upon 
uncovering physical or sexual abuse or if Oasis Center is unable to locate an 
Emergency Residential Shelter resident’s guardian. 

Key representatives from these organizations have identified barriers and 
facilitators to interagency coordination. The barriers typically relate to the structure 
and focus of these government agencies. For instance, prior to 1990 Juvenile Court 
was uninterested in taking a proactive approach to the treatment of Court clientele. 
But a newly appointed Judge has shown great interest in using the community 
resources available to at-risk youth. As a result, Oasis Center provided crisis 
intervention training to Juvenile Court staff and regular interagency meetings began 
in 1990 between the Center, Juvenile Court, and DHS. 
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These agencies cite open communication across agencies as a key to successful 
coordination. As a result of regular interagency meetings, Oasis Center created a 
DHS liaison position to respond to its concerns that DHS did not act on emergencies 
in a timely manner. The liaison spends part of each week at DHS serving as an 
information source about Oasis Center’s services and ensuring that needy youth do 
not fall through the cracks at DHS. 

Funding Sources 

Many Oasis Center programs were initially funded with Federal discretionary 
money; the bulk of the Center’s current funding comes from government grants and 
the United Way. In 1990-1991, Oasis Center’s funding came from Federal and state 
government grants (60 percent), the United Way (26 percent), private contributions (9 
percent), and program service fees (5 percent). Eveiy component of Oasis Center’s 
programming receives financial support from several funding sources. This is a 
deliberate strategy adopted by the agency to assure that changes-especially 
reductions or eliminations--in funding sources do not completely wipe out any 
program component. 



Evaluation 

Oasis Center performs a variety of evaluations, the majority of which are 
process evaluations. Additionally, several programs collect client outcome data 
including Horne-Based. Services, ELECT, and Independent Living. Oasis Center plans 
to incorporate outcome-based evaluation into its Early Intervention programs. Ninety 
day follow-ups of the program will include self-reported behaviors, attitudes, and 
knowledge related to the contents of the Early Intervention activities. 
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Many of Oasis Center’s current grants include a mandatory evaluation 
component. Computer-generated data reports are produced monthly to track case 
load, client disposition, and management. The Center has participated in impact 
evaluations for specific service components when the funding source supporting them 
required it. 



Brief History 

The Center for Family Life was established as a replication of a successful 
community outreach program called the Family Reception Center (started in 1972 in 
the adjacent Brooklyn neighborhood of Park Slope). Catholic Charities and the Child 
Welfare Administration (CWA— part of New York City’s Human Resources 
Administration, the name of the city’s social services department) wanted to duplicate 
this model in another needy community. The prime catalysts behind replicating the 
model were two sisters from the Sisters of the Good Shepherd Order, Sister Mary Paul 
and Sister Mary Geraldine. Both remain today as the Director of Clinical Services and 
Center Director, respectively, at the Center for Family Life. The sisters worked at the 
Family Reception Center and had first-hand knowledge of the needs in Sunset Park, 
which ranked among the most impoverished of all neighborhoods in New York City. 
Based on their six-month needs assessment of Sunset Park, St. Christopher’s Home 
made a commitment to open the Center for Family Life. This institution (renamed the 



CENTER FOR FAMILY LIFE IN SUNSET PARK 



Director: 
City/State: 
Phone Number: 



Sister Mary Geraldine 
Brooklyn, NY 
( 718 ) 788-3500 



St. Chris topher’s-Ottilie Home in 1985) is a not-for-profit Long Island child welfare 
agency affiliated with Catholic Charities of Brooklyn and the Federation of Protestant 
Welfare Agencies. It was the original sponsor of the Center for Family Life, and still 
serves as the Center’s fiscal agent since the Center is not incorporated as a 501(c)(3) 
organization. Foundation grants were secured to cover the costs of site renovation at 
an ideally suited central location in Sunset Park and the Center for Family Life 
officially opened in November 1978 to provide intensive family-centered services. 

Since its inception the program has provided both treatment and prevention 
services. It initially emphasized treatment-oriented individual and group casework 
services for families in crisis. In response to community needs, however, the Center 
for Family Life began enhancing their prevention components over a ten-year period. 

In 1981 an Employment Readiness program for adults was initiated through Federal 
funding from CETA and JTPA and operated in the Bush Terminal area of Brooklyn. 

An after-school program and Teen Evening Center was initiated in one local 
elementary school in 1980-81, followed by a similar afternoon program and Teen 
Evening Center at a second school (p.S. 314) in 1983. In 1991 an afterschool center 
was opened at a middle school to complete the current service array in three schools. 
The two Teen Evening Centers and three afterschool programs, in three schools, 
become summer day camp programs in July and August of every year. 

In 1980 the Center for Family Life was a prime mover behind the formation of a 
Human Services Cabinet to bring together all service providers in Sunset Park (this 
Human Services Cabinet is described in more detail under service integration). Also 
in 1980 the Center initiated a storefront Thrift Shop, Advocacy Clinic, and Emergency 
Food Program in collaboration with other community agencies; these programs moved 
to their present storefront site in February 1989. In 1983-1984 the teen programs 



O 

ERLC 






were expanded to include a Counselor-in-Training program that developed youth 
leadership and mentoring capabilities among younger adolescents, who were later 
hired to assist the after-school program for the younger children. In 1989 the Center 
for Family Life took over the community’s Summer Youth Employment program when 
that program was about to be terminated. In 1991 the Center for Family Life 
successfully obtained a grant through a Dewitt-Wallace/Reader’s Digest School 
Partners Project to develop its third school-based arts enrichment and afterschool 
center program-the first to be situated in a local middle school rather than an 
elementary school. 

Current Mission, Goals, Objectives 

The mission of the Center for Family Life is to provide an integrated and full 
range of personal and social services to sustain children and families in their own 
homes, to "counter the forces of marginalization and disequilibrium which impact on 
families," to buffer the negative influences of the environment on children, youth, and 
families that lead to delinquency, and to provide alternatives to foster care or 
institutionalization. The Center meets this mission by providing a broad spectrum of 
recreational, enrichment, supportive, and counseling services to children, youth, and 
families living in the Sunset Park neighborhood. A further goal is to make changes 
not only at the individual and family levels, but also at the community level. The aim 
is to help the community develop, through its own efforts, the services and activities it 
has identified as needed. The Center emphasizes empowering community members to 
address community needs collectively. The Center sees itself as a combination 
settlement house, child guidance clinic, and community center that holds to the 
principles of providing a broad continuum of services in a non-labeling, non- 
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stigmatizing, and non-categorical fashion. Its objectives are to foster access to 
normalizing opportunities, build competence, resolve conflicts and crises in families, 
change the underlying environmental conditions affecting family and community life, 
and engage in inter-organizational planning and exchange to promote collaboration of 
all human service agencies in the community. 

Service Configuration 

Families, children, and youth initially come to the Center in either of two ways: 
as a family casework client or as participants in the Center’s open programs. Families 
who enter as registered casework clients receive intensive short- or long-term 
counseling for family crises in order to reduce the risk of serious long-term problems 
or family breakup. These families can either seek services themselves or can be 
referred from the district Department of Social Services, New York City’s Child Welfare 
Administration (a public agency), school guidance counselors, or school principals. 
Families, children, and youth who come to the Center as open participants are not 
generally referred by an outside agency or service provider and typically do not have 
identified service needs but simply want to participate in the Center's enrichment and 
recreational programs. Any resident in the Sunset Park community is eligible to 
participate in the open programs. 

The Center offers families in the family casework program a wide range of 
support and counseling services and activities. Comprehensive assessment and 
evaluation services assist in developing an individualized treatment plan for the 
family. The Center offers short- and long-term counseling using individual, group 
and/or family sessions as appropriate to the particular family. The counseling 
services may involve more than one method of therapy and may include as many 



family members as required. As adjuncts to the counseling, families also participate 
in family life education and discussion groups, women’s support groups, and 
therapeutic activity groups for children and teens. There is also an in-home aid and 
support service provided through a Foster Grandparent program in which elderly men 
and women, supported by the Center’s professional counselors, visit the home and 
give support to parents and families in crises. Casework families can also obtain 
medical, legal, vocational, social, and religious assistance through other community 
agencies and services. Families also receive extensive help in assessing and 
remedying school problems and learning disabilities. For these activities, Center staff 
work with school-based support teams and share evaluation and planning duties with 
school personnel in developing an individualized educational plan to move the child or 
youth toward mainstreaming. Families requiring emergency food or clothing have 
access to the Thrift Shop, Advocacy Clinic, and Emergency Food Program. The Center 
also supports and licenses a small number of satellite foster family homes that 
provide care for neighborhood children in instances of serious crises, so that children 
and youth do not need to be removed from their own neighborhood, schools, friends, 
and other close ties. Keeping the child in the neighborhood also facilitates more 
intensive services aimed toward family reunification, thereby preventing long-term 
out-of-home placements. 

Both casework and open-enrollment families have access to a broad array of 
preventive and enrichment activities. The Center provides comprehensive, enriched 
school-age child care and extended day activity programs on-site at two elementary 
schools in the community. Programs include dance, drama, arts and crafts, sports, 
cooking, and homework help, as well as activities for parents. The after-school 
programs at the elementary schools involve teenage counselors and counselors-in- 
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training as leaders and mentors for the younger children. 

Casework and open-enrollment families can also take advantage of the Teen 
Evening Centers (open two evenings per week at each of two public schools) which 
offer a range of recreational and enrichment activities as well as specific preventive 
and teen leadership programs. At a third (middle) school. Center staff operate an arts 
enrichment program in a number of classrooms and an extensive afterschool program 
consisting of a learning center and activities in theater, dance, visual arts, and other 
arts. The highlight of each of these three school- based programs is an end of the year 
school-wide theatrical performance for the school and community in which all youth 
who participate during the year take part. 

An Infant/Toddler/Parent program provides early stimulation and group play 
for infants and toddlers 6 months to 3 years of age. Children are supervised by early 
childhood teachers while mothers meet in an adjacent room in group sessions as a 
support to each other in resolving personal and parenting needs. Parent workshops 
and community forums on a variety of topics are organized at nearby public schools 
and other sites during the school year. Workshops are held in three languages: 
English, Spanish and Chinese. Finally, a Parent Advisory Council was created to 
provide policy and planning advice to the Center. 

All families also have access to two employment training programs. One, the 
Pre-Employment Services and Job Placement Program, is designed primarily for 
parents. It provides counseling, job search assistance, and job placement for adult 
men and women. The second employment-oriented program is the Summer Youth 
Employment Program, funded by the city’s Department of Employment. The money 
for both of these programs comes through city agencies; the Center is unclear about 
whether the money includes Federal funds. The Center recruits all teens for the 



program from among youth aged 14 to 21, places them with cooperating non-profit 
organizations in and near Sunset Park, and offers the youths concurrent workshops 
throughout the summer on sexuality issues, career planning, and multi-cultural 
relations. Approximately 30 organizations accept SYEP teens and each year; each 
participating agency maintains records of the youth’s attendance on the job and 
provides job coaching and guidance to prepare the youth for future labor market 
participation. More than 700 youth participated during the summer of 1992. 

Current Clientele/Users 

Center programs serve children and youth (from birth to 18 years) and their 
families. Any resident of the Sunset Park community is eligible to participate in the 
open-enrollment programs, since these enrichment and prevention services define risk 
according to the antecedent condition of living in the neighborhood in which there is a 
high rate of poverty, overcrowding, intra-familial disruptions, and social isolation 
reinforced by language and cultural differences. In its 1992 annual report, the Center 
described the race/ ethnicity of children and youth in the open-enrollment programs 
as follows: 81 percent are Hispanic, 8.9 percent are African-American, 3.7 percent are 
Asian, 2.6 percent are white and 4 percent come from other groups. Slightly more 
male than female children participate in the programs (55 percent male) and 50 
percent of all children are between the ages of 10 and 15 years, with the remainder 
about equally split between those less than 10 years of age and those 16 to 20 years 
of age. 

The Center has established two criteria for eligibility for its casework services, 
based upon its desire to make itself accessible to community families as a generic 
family support agency without the formal screening processes and potential stigma 
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families might feel in applying for government programs. The two criteria are that the 
family reside within the Sunset Park neighborhood and that the household unit 
include at least one child under the age of 18 or a pregnant woman. Families 
receiving counseling and intensive casework services are generally those whose 
children are considered at significant risk for removal from the home due to a variety 
of intra-familial and/or environmental problems. 

Under the terms of the Center’s contract with the city’s Child Welfare 
Administration, authorized by New York’s Child Welfare Reform Act, the program is 
obligated to serve at least 29 families in any month (and 55 families over the year) 
who are directly referred by the Child Welfare Administration because of documented 
neglect or abuse. In addition the contract obligates the Center to serve a minimum of 
187 additional families annually who either refer themselves or are referred from any 
other source. Originally the Child Welfare Administration funded the Centers 
casework services to meet three goals of the Child Welfare Reform Act: preventing 
foster care placement of children in those instances in which the risks can be 
managed within the home and community ("unnecessary" foster care); facilitating the 
return of children already placed in foster care: and averting the return of children to 
out-of-home placements (recidivism). The Center for Family Life augments these legal 
mandates with its own broader goals for casework services. The Center approaches 
all presenting problems of children and youth through a family focus; it directly 
provides or arranges for a range of therapeutic interventions to meet the needs of the 
whole family, which in many instances it assesses as underlying the particular 
problem exhibited by a child or youth. 
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Type and Makeup of SI Network 

The Center for Family Life engages in wide-scale, comprehensive, and well- 
planned service integration efforts that take staff and resources directly into the 
community. Its SI network operates both in-house, off-site at other agencies, and 
through informal arrangements with other agencies and organizations. Its in-house 
service integration involves accepting referrals for counseling services from child 
welfare and social service agencies, the courts, and the school system. In addition, it 
operates workshops and family life enrichment groups for casework and open- 
enrollment families at its central building. 

The bulk of its programs are delivered off-site at other agencies, particularly in 
local schools. The afterschool care program operates in two elementary schools and 
one middle school in the community. The arts enrichment program is conducted by a 
Center for Family Life staff person in several classrooms at a local middle school. This 
staff person also sits on a sub-committee of the school’s site-based management 
committee which deals with the coordination of services by community agencies at the 
middle school. The Teen Centers are run at two public schools, two evenings a week 
at each site for a total of four evenings weekly, by Center for Family Life staff and teen 
leadership volunteers and counselors. In addition, Center for Family Life caseworkers 
meet regularly with the guidance counselors and school staff to initiate and monitor 
individualized service plans for students with academic or behavioral problems. 

In general, the Center eschews formal agreements with the schools or any other 
collaborating agencies in favor of more informal arrangements. Center staff meet with 
school personnel to plan activities and programs that meet the school's needs and 
that can be operated within school guidelines. Their experience has been that the 
local schools have so many needs that they welcome anything the Center proposes 
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and. without formal written agreements, will provide Centei staff with direct access to 
children and youth in the classrooms as well as outside of school hours. 

Three other collaborative efforts exemplify the highly developed cooperative 
ventures in which the Center and other agencies participate without benefit of formal 
written agreements. The first of these is the Thrift Shop, Advocacy Clinic, and 
Emergency Food Program that the Center operates in conjunction with many other 
community groups, churches, and not-for-profit agencies. Before this program began 
in 1980, the Center and other agencies had identified a need for this particular set of 
services, and for an easily accessible, informal, and non-stigmatizing mechanism for 
delivering them. The collaborating agencies each contribute goods and services to 
these programs, which are available at a storefront location. The emergency food 
bank, for instance, is stocked through periodic food drives (and sometimes through 
direct purchase). When a client family of any of the participating agencies need food, 
the agency gives the family a voucher which tin- family takes directly to the food bank 
and exchanges for groceries. Now in its twelfth year, the program flourishes without 
benefit of any written commitments among agencies. 

The Center operates the Summer Youth Employment Program under a contract 
with the city. The Center must locate and work with approximately 30 non-profit 
agencies, each of which provides summer jobs for one or more youth. Each of these 
host agencies must complete a written application to participate in the program. The 
application states the number of youth the agency will accept and the number and 
types of assignment available (e.g., clerical, advocacy). Other than these agency 
applications, there are no other formal agreements between the host agencies and the 
Center for Family Life. Under the program the Center recruits, screens, and places 
the youth in agencies and offers a variety of support activities during the summer. 
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The host agencies supervise and work with each youth to develop attitudes and habits 
that will lead toward future labor force participation. Upwards of 700 youth 
participated during the most recent program year. 

The third informal but highly collaborative arrangement in which the Center 
participates is the Human Services Cabinet. The Human Services Cabinet is 
comprised of representatives of about 60 public and voluntary agencies and 
community groups in the Sunset Park area. The Cabinet is an arm of Community 
District Board #7. New York City is divided into 59 Community Districts, each 
administered by a Board that is part of the city’s governance structure. The 
Community District Boards are intended to bring resolution of local matters more 
under the control of local community members. The Human Services Cabinet is 
designed to coordinate services and to plan for community-wide events within 
Community District #7, and also to initiate timely responses to emerging 
neighborhood issues affecting families and children. All agencies and organizations 
operating in the district can become members of the Cabinet, and to date more than 
60 of them participate. There are no formal documents of membership, nor are there 
formal decision-making processes. The Center for Family Life often acts as both 
opinion leader and catalyst for planning within the Cabinet. 

The Human Services Cabinet has evolved over the twelve years of its existence. 
Agencies are usually represented by their directors or high level staff. It produced a 
resource directory of the 60+ agencies in the District to improve interagency referrals. 
It is a forum for discussing issues that affect the whole District and its agencies. 
Member agencies are beginning to work on joint grant applications to meet needs 
identified through the Cabinet. It makes recommendations to the Youth Committee of 
the Community District Board, which has some resources to allocate. Allocations 
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have begun to reflect the recommendations of the Cabinet in recent years. The 
Cabinet tries to increase the comprehensiveness of services and activities in the 
community by identifying needs and working together to develop plans to meet those 
needs. 

Funding Sources 

The Center obtains nearly 70 percent of its funding from the public sector and 
relies on grants and contributions from foundations, corporations, and individuals for 
the remaining 30 percent. Staff philosophically oppose receiving public funds from 
categorical or single-problem funding streams. The treatment services provided by the 
Center, in the form of counseling and casework, are relatively well-funded, while the 
more prevention-oriented open-enrollment programs appear to suffer from unstable 
and inadequate funding. One-half of the Center’s budget goes toward support of the 
counseling and casework services. The Center receives the bulk of its funds for these 
programs from the Child Welfare Administration (a combination of city and state 
funds). The New York City Department of Employment funds the Summer Youth 
Employment Program and the adult Pre-Employment and Job Placement Program, 
although a portion of the program’s funds come from JTPA. Together these public 
agencies supply about three-quarters of the funding for the employment programs; 
the rest of the funding is private. Finally, the New York City Department of Youth 
Services provides one-half of the funds required to operate all school-based services, 
including the after-school program. The New York City Department of Youth Services 
has a private match requirement. 

Of the Center’s many programs, the school-based prevention and enrichment 
oriented programs (afterschool programs and Teen Evening Centers) are the biggest. 



serving more than 2000 children and youth annually. They are also the most 
vulnerable to funding cuts because they rely most heavily on support from 
foundations and private donors (public sources supply only about half of their annual 
operating expenses). These private sources are more likely than public programs to 
change their priorities, to limit each grantee to only a few years of support, or to 
require new services in exchange for continued support. A host of foundations 
provide funds to support Center activities, including the Foundation for Child 
Development, the Morgan Guaranty Trust Company Foundation, the Robin Hood 
Foundation, and the Tiger Foundation. IBM donates computer hardware and 
software. 

Evaluation 

The Center maintains extensive records for its casework families, as required by 
the Child Welfare Administration. Detailed statistics are kept on every case, and 
cumulated each month in a report to CWA. At the end of the year the Program 
Director aggregates the statistics for program use in annual Progress Reports and for 
program planning. 

In a program with as many inter-relating components as this one, it is not 
surprising that one difficulty encountered is being able to account for all services 
given. There are concerns both about the amount of time it takes to document service 
activity and to aggregate the data, and about possible under-reporting of service 
delivery in some program components. Staff feel that having workers maintain timely 
and complete documentation of all service delivery can be very taxing. Since the 
Center is Just starting to computerize its records, staff use manual spreadsheets to 
record service use. They also must aggregate much of the raw data by hand. The 
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staff report that having so many different services makes it difficult to assure that all 
services from all components are recorded. For example, there is a suspicion that the 
use of food vouchers at the Thrift Shop and Emergency Food Bank is under-reported. 

Documentation appears weakest in tracking clients and services in the 
prevention or open-enrollment programs. Because it is interested in documenting the 
impact of the Center’s prevention efforts, the Foundation for Child Development is 
helping the Center set up a database to track child and parent use of prevention 
programs, especially the afterschool programs, Teen Evening Centers, and Parent 
Council. Only within the last two years has the program been able to generate a list 
of unduplicated cases for various programs; the list goes to the Department of Youth 
Services (the funding source for the afterschool programs). 

Although they do not actively conduct extensive evaluation research. Center 
staff have some experience with various forms of evaluation, including: 

• The Child Welfare Administration conducts a yearly quality assurance review 
that consists of a site visit and selected case reviews. The CWA official reads 
selected case records, evaluates the action taken, and determines whether it 
meets performance standards. 

• The Center conducts an annual client satisfaction inquiry, sending client 
satisfaction questionnaires to all families who have closed their casework 
involvement during that year (plus a sample of open cases). The questionnaires 
are anonymous and include self-addressed, and stamped return envelopes. 
Results are used to review and improve service delivery. 

• The Foundation for Child Development funded an evaluation of the Center’s 
adult employment program to look at child and family issues related to job- 
taking. The Foundation’s concern is to understand the impact of the welfare- 
to-work transition on the ( hildren of the household and on family functioning. 
By late 1992 the Center completed intake assessments of the entire sample of 
150 parents and children; it vvill conduct follow-up interviews with the same 
families every six months for 2 years. 

• The Center also participated in an evaluation of how its work is perceived by 
the larger community of Sunset Park, carried out by a researcher engaged by 
the Surdna Foundation. 

• The Center is now one of eight finalists in the Annie E. Casey Foundation’s 



search for four family support programs to participate in a national evaluation 
study. Its negotiations with the Foundation have included discussions of what 
outcome measures the Center considers adequate to reflect both program 
impact and client experiences in the program. 

The staff of the Center for Family Life are interested in evaluation research and 
have already participated in a variety of evaluation and assessment activities. They 
are eager to participate in a research project whose design they consider adequate, 
and with appropriate staff support. They also have some concerns about evaluation 
research. They are concerned that staff time spent on evaluation activities would not 
be reimbursed. They are also wary of evaluation designs in which outcome 
measurement is either simplistic or makes questionable claims given the data 
available. They are, however, very interested in participating an research based on a 
solid design and employing evaluators trained in social work research and outcome 
measurement in particular. 
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TEEN CONNECTIONS 



Program Director: 
City/ State: 
Telephone Number: 



Myraia Bass-Hargrove 
Bronx, NY 
( 212 ) 590-4050 



Brief History 

The Girls Club of New York’s Teen Connections program is part of a national 
demonstration designed to "...(improve) the health of early adolescent girls, especially 
those at high risk of developing poor health behaviors." Established in 1990, Girls 
Club of New York was one of four affiliates selected by Girls Inc. to participate in the 
three year pilot project funded by the Kellogg Foundation. The impetus behind the 
program was a perceived decline in the health and physical fitness of adolescent girls. 
From its inception, the program has been prevention-oriented; its primary treatment 
activity has been referring program participants to the appropriate community 
agencies for treatment. Although the Kellogg Foundation provides the funding for the 
Teen Connections demonstration, it has not supported the Body-By-Me component. 
This component is funded by the city of New York’s Department of Youth Services. 

The program has evolved continuously. Program objectives have been refined 
and revised as has the working definition of "at risk." During its first operational year, 
many of Teen Connections’ school-based referrals were high school dropouts with 
many problems. The program found that most of these girls needed far more support, 
assistance, and intervention than the program’s prevention activities were set up to 
handle. In subsequent years, therefore, the program set up a screening process, 
refining its risk definition to insure that referrals are more appropriate for the 
program’s prevention goals and services. 

It is important to understand the structure in which the Teen Connections 



program operates and the interdependence of the organization’s multiple layers. 

Girls, Inc. is the national organization that received the Teen Connections grant and is 
ultimately responsible for providing each site with the budgetaiy support to operate 
Teen Connections. Girls Club of New York, the site of the Teen Connections program 
studied, is an affiliate of Girls Inc., although it has not been actively involved in the 
core programming sponsored by Girls Inc. Girls Club of New York has been interested 
in dealing with teenage health issues for some time, so Girls Inc. perceived the Girls 
Club of New York to be well-suited to participate in the Teen Connections 
demonstration . The Girls Club is in the process of reevaluating its mission; the 
resulting uncertainties about where its parent organization is going affect the mission 
and long-term viability of Teen Connections. 

The Girls Club of New York’s Board of Directors also serves as Teen 
Connections’ Coordinating Council. The other Teen Connection demonstration sites 
have enlisted outside experts to be members of their Coordinating Council. Because 
the program’s parent organization, Girls Club of New York, has been without an 
executive director for over six months, the role of the Board of Directors has expanded 
to ease the agency’s increased burden. In addition, Teen Connections experienced the 
loss of its first program director within one month of the program’s inception. A four- 
month search ensued before hiring the present director, during which the program 
was without direct management. The present director has been with the program for 
the past three years. As a result of staff turnover and uncertainties within the parent 
organization, Teen Connections has been forced to operate in a somewhat 
unpredictable environment. 
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Current Mission, Goals, Objectives 

The Teen Connections mission is to improve the well-being of youth in the 
South Bronx through application of a holistic view of each youth in service delivery. 
The program’s major goal is to "...train teens to meet their own health needs. ..through 
a comprehensive preventive approach that includes case management, peer 
counseling, health, fitness, nutrition, and teen directed community health projects." 

Teen Connections has a detailed set of program objectives that reflect its 
diverse components. These objectives include: 

Health Fair 

• Outreach to and network with a broad spectrum of service providers. 

• Provide an opportunity for teens to interact with service providers on the teens’ 
turf. 

• Recruit teens for Teen Connections and other Girls Inc. programming. 

• Increase teens’ knowledge about their health. 

Connections Advocacy (case management component) 

• Outreach to and recruit high risk teens who would not normally come through 
the doors: local definition of ’high risk’ is to be documented by the affiliate. 

• Serve existing members in need of support. 

• Provide supportive services through groups, one-to-one sessions and referrals 
to internal and external resources as needed. 

• Identify gaps and inadequacies in services to contribute to solutions that strive 
for change. 

• Involve significant others as needed to meet the needs of teen participants. 

• Intentionally integrate Connections Advocacy with the other components of the 
project and the rest of the organization. 

Teens for Teens 

Help teens understand health issues and how the issues impact on teens. 
Develop the leadership skills of teen interns (ages 15 to 18) for the purpose of 
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engaging and developing the leadership skills of 12- to 14-year-old teens. 

• Provide opportunities through community action projects for teens to take 
leadership roles around health issues. 

• Facilitate teens’ identification of a problem and implementation of a plan for 
change through a community action project. 

Body By Me 

• Develop and implement a program which: 

Provides at least 30 minutes of health- related, cardiovascular fitness-building 
activity in the physical activity portion: 

Increases teens’ knowledge of health and fitness issues, covering the areas of 
physical fitness, nutrition, substance abuse and stress management: 

Provides nutritious snacks for teen participants: 

Is based on sound, expert knowledge. 

• Help teens develop and engage in realistic options for improving and/or 
maintaining their health and fitness. 



The major objectives to be achieved by the program’s Coordinating Council 
include: 



• Identify existing community resources to be included on the Coordinating 
Council. 

• Utilize the support, advice and expertise of community resources. 

• Outreach to and network with service providers. 

• Act as an agent for change. 

The program has evolved continuously and some of its goals and objectives 
have been modified and streamlined to meet needs uncovered at the Teen Connections 
pilot sites or changes introduced by Girls Inc. or the demonstration’s funder. For 
instance, although the "involvement of significant others" has always been an 
objective of the Connections Advocacy program component, the funder placed 
increased emphasis on this objective during the third year of the demonstration. The 
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demonstration protocols had always required parental consent for youth to 
participate, but had not always received it. Now the program places more emphasis 
on counseling the youth to get this consent from their parents. But the program’s 
service emphasis remains on the youth. In response to the wishes of the Kellogg 
Foundation, the Center emphasized the role of the Coordinating Council as "an agent 
for change" and, with it, the notion of systemic change. 

Service Configuration 

Teen Connections offers prevention and case management services. The 
program has four distinct components: Body By Me, Teens for Teens, Connections 
Advocacy, and Health Fair. Clients may participate in any of these program 
components and need not enter through case management. All of the Teen 
Connections activities occur at the agency and case management is also conducted at 
two school sites. 

In order to participate in case management activities, potential participants and 
a parent must sign consent forms. For case management clients, service delivery 
involves an extensive assessment, including questions about the participant’s 
nutrition, dental care, education, drug habits, home life, mental health, health needs, 
etc., and the development of a case plan. Youth must exhibit one of the specified 
health risk factors in order to participate in the program’s case management 
component. The risk assessment lets the case manager identify inappropriate cases, 
prioritize cases, and identify the clients that require immediate referral but not long- 
term case management. Those individuals who simply desire to participate in the 
program’s health and fitness component, Body By Me, typically are not included in 
the case manager’s case load. Caseload clients meet regularly (every week or two) 



with the case manager although the frequency of these meetings is situation-specific. 
The primary responsibilities of the case manager include providing health information, 
making client referrals, and encouraging clients to learn about health-related 
community resources. 

While the case manager makes referrals, it is the client’s responsibility to set 
up and attend appointments— in essence, to ensure that their own health needs are 
met. There is an attempt to involve parents in a child’s treatment plan if the child is 
amenable. Participation of a child’s significant others is mandatory only in cases that 
involve imminent danger. To obtain assurance that the youth received the service, the 
case workers ask the youth themselves and routinely contact the referral service both 
before and after the expected service contact (to tell the referral agency to expect the 
youth, and to see whether the youth got there). 

Body By Me. The twelve week Body By Me curriculum is offered twice each 
year to individuals aged 12 to 15. The program’s primary focus is nutrition but it 
addresses a variety of health- related topics, including: 1) communication, 2) self 
esteem, 3) hygiene, 4) substance abuse, 5) stress management, and 6) teen sexuality. 
The program meets three times each week and its weekly structure includes fitness, 
health education, and recreational activities. Participants can enter any time during 
the twelve week cycle if there are openings but class sizes are limited to approximately 
25 to 35. 

This component has been modified significantly during the course of the 
demonstration. The program added recreational activities to the original curriculum 
structure as an incentive for program participation and modified the health education 
curriculum was to include adolescent sexuality— a topic of importance to many 
participants. 
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Teens for Teens. Teens for Teens focuses on leadership development and 
community action. The program is structured in two 12 week phases that incorporate 
training of high school aged teen "leaders" and field experience. Teen Connections 
staff recruit high school students who are interested in health issues (there are three 
teen leaders this year) to participate as youth role models as they design and 
implement a community action project. These teens receive training (e.g., instruction 
in project planning and working in groups) and a certificate of leadership. The teens 
are then responsible for recruiting students between the ages of 12 and 16 who are 
interested in participating in the program. While the primary focus is on recruiting 
female participants, males are accepted into the program. As a group, they design 
and carry out community-related health projects. During year two of the 
demonstration, Teens for Teens participants created a public service announcement 
on AIDS. 

Connections Advocacy ( case management). Youth aged 12 to 15 have access to 
Teen Connections’ case management services. The case manager maintains an office 
at the Girls Club but provides much of the case management at selected school sites. 
There is no active recruitment of clients and a school nurse or other staff member 
typically refers participants. Youth alsG learn about Connections Advocacy through 
informal contacts with other agencies and word-of-mouth. However, the case 
manager uses Connections Advocacy as a vehicle from which to advertise the other 
components of Teen Connections. The program has one case manager for its two 
school-based sites. The case manager typically sees approximately 15-20 cases 
throughout the school year and 10 cases during the summer. The program has had a 
number of longer-term cases, although most clients simply request some type of 
health information. 




i-L ^ « > 



195 



Health Fair. The annual Health Fair exposes participating teens to information 
about local health agencies and attempts to promote the use of the agencies’ services. 
Teen-to-Teen youth participants have the responsibility to recruit youth presenters for 
the Health Fair, but an organizing committee of adults has overall responsibility for 
the event. 

Current Clientele/Users 

Teen Connections serves 12- 18-year-old males and females residing in the 
South Bronx. The program’s case management component serves individuals aged 
12-15. While the program’s primary focus is on the health issues facing adolescent 
girls, the needs of the entire community are so great that programming is also made 
available to local boys. Approximately eighty percent of its program’s participants are 
African-American and twenty percent are Hispanic. Attendance in the specific 
programs is limited; at any given time Teen Connections serves approximately 15-20 
youth during the school year and 10 during the summer months through case 
management. 500 at the health fair. 25 to 35 with Body By Me. and 20 younger 
participants and several teen leaders through Teens for Teens. The program keeps 
groups small so staff can pay more personalized attention to each participant, but this 
year it is experimenting with groups of up to 50 youth. The racial composition of the 
Body By Me and case management components varies each time they are offered but. 
on average, African-Americans and Hispanics participate equally. Most clients come 
from low income single parent families. Youth become aware of the program through 
word-of-mouth or referrals by school personnel or other professionals for case 
management. A concern voiced by participants and staff was the lack of widespread 
knowledge about Teen Connections within the community. This "image problem" is 
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exacerbated by the fact that Teen Connections is housed in the Girls Club, which 
many youth do not perceive is a place to go for the type of program offered by Teen 
Connections. This is the first Girls Club program that attempts to reach a broad- 
based clientele. The program Is working to change these perceptions. 

Any interested youth within the specified age range may participate in Body By 
Me. Participation in the other program components is more restricted. Teen leaders 
are recruited and must go through an application and screening process. Those teens 
who express an interest in health-related topics are favored. Youth must exhibit one 
of the defined risk factors in order to become a "case" in Connections Advocacy. 

These risk factors include: 

• Excessive absence or restriction from activities due to a health problem. 

• Poor appearance. 

• Over or under weight. 

• Involvement in behaviors that put the youth at risk for teen pregnancy, AIDS, 
or sexually transmitted diseases. 

Type and Makeup of SI Network 

Teen Connections has no formal contracts with other agencies. Most of its 
activities and services are offered in-house. The program does have informal linkages 
with two junior high schools and a number of local agencies. The case manager 
obtained permission of two junior high school principals to locate Teen Connections’ 
case management function within the schools. The case manager shares an office 
with a school staff member while in school. Additionally, since Teen Connections does 
not have the capacity to perform home visits, the schools’ dropout prevention 
programs conduct these visits and may refer some of these youth to Teen 
Connections. 
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Teen Connections has also formed relationships with key community agencies 
and youth are referred to these agencies for services, as needed. One of Teen 
Connections’ major referral agencies is the area’s Planned Parenthood office, the HUB, 
which provides services related to pregnancy and sexual activity. Other agencies to 
which Teen Connections refers clients include the Fordam-Tremont Clinic (for mental 
health services) and the Citizens Advice Bureau (geared toward parents who seek 
information about domestic violence issues). 

Teen Connections has recently forged a relationship with Lehman College. The 
college has provided a number of health education interns who will work with Teen 
Connections staff to support its programming. Teen Connections has also attempted 
to work collaboratively with other Girls Club of New York programs. During the 1992- 
1993 program year, Teen Connections had hoped to collaborate with two additional 
Girls Club programs— the Options Center, which offers a violence forum, sports, and 
fitness activities, and the Youth Employment program. Teen Connections and staff of 
these two programs developed a plan for how this collaboration will proceed, but it 
has not yet begun. The plan includes sharing staff and financial resources among the 
programs in order to extend the Club’s hours and give youth access to a broader 
range of services. 

Funding Sources 

The Kellogg Foundation is the primary funder of this demonstration project, 
although the city of New York’s Department of Youth Services provides the financial 
support for the Body By Me component. The Kellogg Foundation awarded $1.8 
million to Girls Inc. for the four year demonstration (3 years are operational and 1 
year is administrative). The national evaluation headquarters is at Girls, Inc. in New 



York, but the project director for Teen Connections is in Indianapolis. The national 
program regulates the flow of funds to the local demonstration sites and New York’s 
Teen Connections receives approximately $100,000 each year to administer the 
program, which it spreads fairly evenly across the sites. 

Teen Connections program staff indicate that they unsuccessfully tried to 
obtain additional funding for their Bronx program to meet operational expenses but 
that Girls Inc.’s "pass through" system of allocating funds did not allow funding- 
related modifications in response to site-specific expenses. The Kellogg Foundation’s 
demonstration funding ends at the conclusion of the 1992-1993 program year and 
continuation funding has not yet been secured. 

Evaluation 

Teen Connections-Bronx participates in a national evaluation along with the 
three other Teen Connections demonstration sites. However, the Bronx Teen 
Connections site and one of the other three sites have had some difficulties with the 
independent evaluator conducting the national data collection effort. The Bronx 
program reports that some clients were offended by specific questions on a form the 
youth had to complete themselves (one of the questions asked the teens if they had 
bugs in their house), and that the evaluator made remarks in the hearing of 
participants that reflected a disrespectful attitude (the evaluator was overheard 
commenting that the kids were "making babies in the school halls"). One of the other 
evaluation sites had similar experiences. The national program staff were made aware 
of these issues. Arrangements have now been made for these two programs to 
continue with the national evaluation, but for the evaluator to refrain from certain 
types of direct contact with the youth. Teen Connections still sends monthly program 
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reports to the Girls Inc. national director. Teen Connections performs basic record- 
keeping and has access to data from a variety of forms, including: 

• Case management referral forms, including information on referral reason and 
an assessment of risk indicators. 

• Intake forms, including client background information. 

• Service referral forms, including information on referral type, provider, date of 
referral, and date of service (obtained from follow-up calls the referral agency 
and by asking the youth directly). 



Brief History 

Houston Communities in Schools (CIS) began in May 1979 as the first Texas 
implementation of the national Cities in Schools program model, which was based on 
the late- 1960s "street academies" for poor urban youth. Originally named Houston 
Cities in Schools due to its affiliation with the national Cities in Schools organization, 
the Houston CIS left the National Cities in Schools organization in 1986 and became 
incorporated as "Communities in Schools." 

The Houston CIS was initiated by Juvenile Court Judge Wyatt Heard in 
conjunction with the Houston Independent School District (HISD), the Chamber of 
Commerce, the Houston Mayor, and various community and business leaders. The 
first CIS site opened in 1979 at M.C. Williams Middle School in the Acres Homes area 
of Houston, with a first year budget of $80,000 and two full-time paid employees (the 
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Executive Director and the Project Manager at the school site). The first external 
grant came from the State Department of Criminal Justice to conduct a dropout 
prevention program. A host of local businesses Including oil. utility, and real estate 
companies contributed funds during those early years. CIS established an 
Interagency Council comprised of all agency partners as well as representatives from 
the school district and business/community leaders to guide policy. Eleven agencies 
participated in the collaboration during the first year of operation including: the city 
health department, parks and recreation department, police, Depelchin Children’s 
Center (serving teen mothers), the Houston child guidance clinic. Big Brothers/Big 
Sisters, and Community Youth Services. 

During the early years, the Houston CIS was supported by grants from the 
state departments of Criminal Justice and Education (Chapter 2 funds) and private 
funds. In 1985-86 a state-wide CIS office within the Governor’s office, supported by 
the Governor’s Discretionary fund, was created and headed by Jill Shaw, the original 
Houston CIS Executive Director. Cynthia Clay-Briggs, the first Project Manager at 
M.C. Williams Middle School, then became Executive Director of the Houston CIS and 
is currently the Executive Director. The State CIS office is now housed within the 
Texas Employment Commission (TEC). Since 1986 the CIS program model has been 
institutionalized by state legislation as part of the TEC and the Texas Youth 
Commission. In 1990 CIS started receiving compensatory education money diverted 
from school districts. 

Over the years, new school sites were added as money became available and 
demand from principals grew. By the start of the 1989-90 school year, the program 
had expanded to nine sites in elementary, middle, and high schools. Rapid expansion 
has continued over the last three years. In the 1992-93 school year, Houston 



Communities in Schools operates at 18 Houston Independent School District schools 
and three schools in adjacent school districts. 

Current Mission, Goals, Objectives 

The Houston CIS defines its mission as coordinating services to at-risk youth 
and their families in such a way that the whole environment and circumstances of the 
youth and family are addressed. The program addresses the multiple needs of at-risk 
students and their families by providing an umbrella under which all social and 
related services are coordinated and available on the school premises, where it is 
easiest to reach many youth. The overall goals of the Houston Communities in 
Schools project are to decrease the dropout rate, decrease delinquency, prepare 
participants for adult work roles, improve school performance, improve school 
attendance, and increase the graduation rate. 

To accomplish these goals, each CIS school site focuses on counseling, 
enrichment, and academics, and also tries to increase parental involvement in school 
activities. Although CIS administrators note that its mission has not changed over 
the twelve years it has been in operation, methods for carrying out its mission and 
accomplishing its goals have recently shifted. CIS used to approach each new school 
site with a generic plan for developing the program at that school. Now when setting 
up a CIS program at a new school site CIS tries to develop a plan geared more 
specifically to the school in question. Thus the types of programs offered and services 
coordinated at each new school are more tailored toward the specific needs of the 



individual school. 



Service Configuration 

CIS provides prevention, enrichment, short-term treatment, and case 
management activities to children and their families at selected school sites. CIS’s 2 1 
sites differ in the types of services available to program participants; in general, 
services vary according to the specific needs of the school campus and its surrounding 
community. CIS is capable of providing participants with a range of services through 
on-site programs and referrals. Student participants and their families may receive 
support services, individual counseling, academic enhancement, crisis intervention, 
parent involvement, information and referral, social services, English-as-a-Second- 
Language (ESL), employment, and enrichment/ recreational activities. 

Most sites offer a wide range of activities and services. Edison Middle School 
adopted a "club" concept in order to introduce activities into a school that had been 
riddled by female gangs. A modeling club, ESL club, Mariachi club, and other after- 
school activities are some of the activities introduced by Edison’s CIS program with 
the goal of building self esteem. The CIS program at Edison has eight major 
components: 1) counseling, 2) academics, 3) enrichment, 4) career awareness, 5) 
health, 6) parent involvement/ parent clubs, 7) employment/pre-employment skills, 
and 8) social services. 

A memorandum of understanding, drawn up at the beginning of the school 
year between each school site and CIS, formalizes the roles and responsibilities of the 
school and CIS staff. Yearly service goals for the sites often depend upon the grants 
they receive as well as perceived community needs. 

Each school has a CIS office that houses CIS staff and agency partners. Staff 
members assess students, match students with needed services and monitor 
students’ academic progress and utilization of these resources from within this office. 
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After an initial assessment, the case manager channels the client into either a 
caseload or non-caseload track. Caseload clients require more in-depth attention 
through weekly meetings with a case manager. Non-caseload clients often receive 
crisis intervention services and typically participate in one or more of the after-school 
activities. 

The CIS office has an open, non-judgmental atmosphere that encourages all 
students in the school to belong to CIS. CIS actively discourages labeling of students 
and has successfully marketed itself to students and their families as a place to go for 
enrichment/ recreational activities, to belong to a group, and to become more 
successful in school, thereby increasing both accessibility and utilization. 

The CIS office is headed by a Project Manager who, in conjunction with the 
School Principal, has overall responsibility for the CIS program at that particular 
school. The Project Manager is the primary coordinator of on-site agency personnel 
and supervises some of each agency’s activities at the school (along with the agency’s 
own supervisor). In addition, the Project Manager supervises one of two CIS 
caseworkers. The CIS caseworkers are the primary contacts and case managers for 
students, particularly those requiring more intensive intervention services. 

Current Clientele/Users 

The potential clientele of CIS at any given school site includes all students at 
the school as well as their siblings, parents, and other family members, although the 
primary client is the student. Students and/or their parents become familiar with CIS 
at a school by word of mouth or during the initial registration period at the beginning 
of the school year. Parents also register their child for CIS so that they can get such 
benefits as clothing and food vouchers and participate in special activities geared for 
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parents. CIS also gets student referrals from any school personnel (including 
teachers, the principal, or a guidance counselor) for reasons related to acting out 
behavior, crisis intervention, poor school performance, grades, or truancy. 

For the purposes of defining levels of risk among students, CIS staff distinguish 
between "caseload" and "non-caseload" students. Caseload students are those at 
higher risk who generally require the more intensive, counseling-oriented services 
and/or ancillary support services to their families, whereas the non-caseload students 
are those who have immediate needs requiring crisis intervention, and/or simply want 
to participate in the school clubs and enrichment activities sponsored by CIS. 

Caseload students usually come to CIS through referrals from a teacher, principal, or 
school guidance counselor while non-caseload students are likely to be walk-ins. 
Caseload students usually have more than one presenting problem and these may 
include: school infractions, acting out, violent or delinquent behavior, physical/ mental 
health problems, drug/alcohol abuse, physical/sexual abuse, attempted suicide, or 
family financial problems. 

All students receive a risk assessment at a relatively early point in their contact 
with CIS. Throughout their membership in CIS they continue to be monitored and 
non-caseload students can become part of the caseload if they experience personal or 
family crises or their school performance declines. Consistent with the non-labeling 
approach, CIS does not formally identify students as "caseload" or "non-caseload" and, 
in general, no stigma attaches to students who belong to CIS. 

Wide variations exist across all 2 1 CIS school sites exist in the socio- 
demographic makeup of the schools and surrounding communities. Two sites visited 
by the authors illustrate this diversity. One, Edison Middle School, is located in an 
Hispanic community that has been termed "Little M ' dco," while another, Key Middle 



School, is located in a low-income African-American community. Despite the wide 
variations in community backgrounds, the common thread is that CIS chooses 
schools characterized by high numbers of students at risk of school dropout. Almost 
all students live in families with less than $12,000 per year for a family of five (at or 
below the poverty line), 60 percent live in single parent homes or with grandparents, 
legal guardians, foster parents, or independently without supervision, and 
approximately one-third come from families where Spanish is the predominant 
language spoken at home. 

Type and Makeup of SI Network 

CIS has established both formal and informal agreements with a range of 
agencies to provide services to both child and family. These services are provided 
either on-site or by way of referrals to outside agencies. The most common type of 
interagency collaboration involves sharing resources, specifically agency staff. Most 
agency staff members have offices at the school where they work with the other CIS 
staff to deal with client concerns. Larger partners often have contractual agreements 
while the smaller or local partners tend to be transient and their support may not be 
documented in reports. 

During the first year of CIS, 1 1 agencies collaborated to provide services at 
M.C. Williams Middle School. Over time, CIS has recognized the need for defining 
each partner’s roles and responsibilities (e.g., expected student case load) and has 
started to use memoranda of understanding to detail its relationship with the host 
school. This helps CIS to tailor the offerings to the needs of the particular site and to 
ensure that CIS does not become overextended (as has happened in the past). 

Local school CIS programs have linkages with different agencies depending 
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upon the perceived needs of the community. Typically, agencies with workers placed 
at the school sites include: community youth services agencies, the state drug and 
alcohol prevention office, juvenile justice agencies, the city parks and recreation 
department, the state employment office, big brother/big sister programs, and child 
guidance and crisis counseling agencies. Also available at the typical CIS school are 
tutoring and mentoring activities provided by local college and high school students, 
and parenting enrichment and parent-focused services including advocacy, 
information, and referral. The Program Manager at each site and the central CIS 
office develop the linkages with each agency or service present at the site. The CIS 
office has a full-time staff person responsible for forging and maintaining agency 
linkages and providing support to CIS school sites in need of particular services. 

At Edison Middle School, for instance, program pardcipants have on-site access 
to a Community Youth Services crisis intervention worker, a drug counselor funded 
through the Texas Commission on Alcohol and Drug Abuse Prevention, two 
caseworkers (one is a volunteer from the Jesuit Volunteer Corps), and 40 tutors from 
the University of Houston. 

Funding Sources 

During its first five years of operation the Houston CIS received funds primarily 
through private donors with a small amount of initial support from the national Cities 
in Schools program. CIS’ corporate office space was donated by Tenneco, a key 
business supporter of CIS. Since its incorporation as Communities in Schools, the 
program has been funded by public and private sources as well as in-kind 
contributions. The state Communities in Schools also provides seed money to start 



up new project sites. 



During the 1991-1992 fiscal year, the program received: 

• 28 percent of its support from public sources (e.g., Community Development 
Block Grant funds, Houston Independent School District (HISD), Texas 
Employment Commission (TEC)]. 

• 31 percent of its support from private sources (e.g., Houston Endowment, 
Tenneco, Cullen Foundation). 

• 40 percent of its support from in-kind contributions (e.g., sixteen repositioned 
teachers from the HISD, one repositioned staff person from the TEC). 

Evaluation 

Houston Communities in Schools has developed and maintains a 
comprehensive Management Information System (MIS). Based at the Centred CIS 
office, the Director of Evaluation supervises a staff of 1.5 key punching/ data entry 
clerks and two full time evaluators. Each CIS site is responsible for completing a set 
of forms related to risk assessment, student and family characteristics, and CIS 
program activities in which each student and family member participate. The site 
Project Manager reviews all forms for completeness and accuracy and then sends 
them to the central office MIS department for data entry and processing. 

A school or agency personnel making a referral to CIS completes a CIS Intake 
Form at the time of the referral. Both caseload and non-caseload students must have 
a completed intake form. Within two weeks of the referral, a Student Assessment 
Form is completed for caseload students, recording basic intake information such as 
reasons for referral and presenting problems. A signed parent or guardian consent 
form is also obtained for all students. For caseload students, CIS holds a staffing 
meeting to determine services, assigns a CIS caseworker, and opens a folder on the 
student. Once the (caseload or non-caseload) student enrolls at CIS, a Student 
Activity Record form tracks the services and activities received by the student and/or 
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family members from CIS or an agency partner on-site. Separate forms record 
participation of students in group or workshop activities; other forms record 
participation by CIS staff, school staff, parents, or other members of the community in 
CIS-run workshops or group sessions. A Termination Report is completed whenever a 
student terminates either from the caseload only or from all CIS activities. This report 
records a number of potential reasons for termination including achieved goals or 
services no longer required, graduated, moved to another school, referred to another 
agency, alternative educational placement, expelled, institutionalized, and dropped 
out of CIS. Students enrolled in CIS can stop receiving caseload services but can still 
remain in CIS to engage in the enrichment and recreational activities. 

The MIS was designed to meet the reporting requirements of multiple funding 
sources, some of whom require line item budget justification as well as monitoring of 
service functions. The MIS produces reports that describe the amount and duration 
of CIS services provided by type of student, family background, or site. In addition, 
the MIS system records funding data and produces reports that estimate the cost per 
student contact hour depending on the types of services provided. Sites receive 
monthly updates of services provided from this system. 

The current Management Information System was limited until recently by its 
dependence on old computer hardware with insufficient storage capacity and outdated 
software, so that the MIS data for prior school years cannot be contained with the 
current school year data. Computer equipment has just been updated to include a 
new file server that can store longitudinal data. The hardware was purchased out of 
general operating revenues. The Board of Directors decided to make this investment 
because it would greatly facilitate producing the types of data that many CIS funders 
increasingly request as part of progress or annual reports. Technical assistance 
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covering which hardware to buy and how to make it fill the program’s needs was 
provided free by volunteers from the business community and by the vendors from 
whom the equipment was purchased. Future plans include upgrading the software, 
networking the MIS and finance office computers, providing more immediate feedback 
to sites, and beginning to do more outcome evaluation studies including longitudinal 
follow-up of past CIS students. The software upgrade involves installing a program 
that will greatly enhance CIS’ capacity to track more than one client per case or cases 
linked by family relationship (as when both youth and parents receive program 
services or more than one child in a family participates in the program). The state CIS 
program has been discussing whether or not it should support development of the 
capacity to do the type of tracking that this software will enable. Houston CIS hopes 
the state CIS program will fund the software upgrade in Houston as a pilot project, 
and then expand to a statewide capability. 
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CHAPTER 6 



CROSS-PROGRAM ISSUES 



In Chapters 2 and 3 we identified a number of issues that informed our 
interviews during site visits. These pertained to programs’ choice of target 
populations; decisions about scope of services and activities and the means to provide 
them, including formal arrangements with external agencies; and a range of 
evaluation issues. In this chapter we summarize the findings from site visits with 
respect to many of these issues, specifically: 

• Clarity about who is and who is not a client. 

• Client risk levels and their implications for program service offerings and for 
evaluation. 

• Program orientation toward strengthening families and/or neighborhoods. 

• Scope and variety of service delivery, and the meaning of comprehensiveness as 
programs see it. 

• Service integration issues, including the scope and variety of networks and SI 
arrangements, history and evolution of SI, perceived impacts, and difficulties 
encountered and ways of handling them. 

• Program choice and tradeoffs with respect to client age range, 

prevention /treatment orientation, activities /services orientation, youth-family- 
community orientation. 

• Evaluation issues, including program interest in and perceived payoffs from 
evaluation, past histoiy of evaluation activities, level of documentation 
currently available, and our perceptions of the feasibility of a multi-program 
evaluation with programs such as these nine we visited. 



Clientele, Intake Procedures, Termination 

It is often difficult to identify who is a client in many innovative programs that 
provide comprehensive services in an SI approach, primarily because the programs 
offer both prevention and treatment services and because they are located in 
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communities, schools, or both. Variability across programs is to be expected, but 
often there may be as much variation within programs as there is across programs. If 
there are many within and cross-program differences in how clients are defined and 
served, then it will be difficult to design an evaluation that can systematically control 
or account for these variations in order to determine impacts. In this section, we 
discuss how programs define their clientele and how clients enter the program, are 
assessed, become involved in the agency, and how clients leave the program. 

All programs we visited employ extremely broad definitions of their clientele as 
participants in the broad spectrum of program components they offer, spanning 
prevention and treatment modalities. Therefore at least some recipients of program 
services cannot be easily distinguished from other members of the community. Due 
to the mix of treatment and preventive services these programs provide, many youth, 
families, or other individuals may come in contact with the program in some way but 
cannot be considered clients for the purpose of evaluating program inputs and 
outcomes. In general, we will clarify how clients are identified by examining the 
processes of entrance, involvement, and departure from the program. 

Point of Entry 

"Point of entry" refers to the method by which a client first comes in contact 
with a representative of the program or first becomes acquainted with the program. 
Programs that offer both preventive and treatment services provide several means of 
entry, depending on the type of service the youth initially was intended to receive. 
Conversely, programs that focus only on preventive or treatment services reveal a 
somewhat more restricted set of entry points to the program. At one end of the 
continuum are programs offering mainly preventive services— I Have a Future and 
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Teen Connections. Entry into these programs requires that the youth or the youth’s 
parent register with the program or sign up through an application procedure. At I 
Have a Future all of the children enter by committing themselves to a 60-day entry 
period, completing an intake assessment and physical exam, and by participating in 
the required entry service modules. At Teen Connections youth enter by completing 
an application form to participate in one of the preventive activities. At the other end 
of the continuum are programs offering primarily treatment services such as Chins Up 
or Oasis Center. Youth or families are referred to these programs either by school 
personnel or by a caseworker. At Chins Up, all youth are referred from another 
agency (generally juvenile justice or child protection/child welfare), although some 
referrals come from special education or alternative schools. At Oasis Center, youth 
or families may also refer themselves by walking in and asking for help with problems. 

Most of the programs in this study provide both preventive and treatment 
services, so they generally have multiple points of entry into the program. Programs 
with multiple entry points often distinguish between "caseload" and "non-caseload" 
clients. This is sometimes done formally as in Houston’s Communities in Schools 
program, where case files are explicitly identified as caseload or non-caseload, or 
informally as in the Center for Family Life in Sunset Park, where cases are not 
explicitly identified but the caseworkers and program staff know whether a given 
youth and family is "caseload" or not. In general, the non-caseload clients are those 
receiving recreational or enrichment activities and have usually come to the Center 
through a registration or enrollment process, while the caseload clients are those 
referred for treatment or counseling by the school or a social agency due to specific 
presenting problems. At the Center for Family Life in Sunset Park, there are clearly 
identified methods for entry based on whether the initial need is for treatment 
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or prevention/enrichment. The Belafonte-Tacolcy Center distinguishes between 
prevention and treatment cases; many of the youth who enter via the prevention 
programs do so at two schools where Belafonte-Tacolcy maintains an active presence, 
or by coming to the Center for recreational activities and/or team sports. It is equally 
easy to distinguish prevention and treatment clients at Oasis Center, Garfield Youth 
Services and Communities in Schools, the other programs with major efforts in both 
prevention and treatment. 

It appears that a characteristic typical of many SI efforts is that youth and 
families can enter the program without necessarily being identified as needing a 
specific service. This is especially true for the more comprehensive service delivery 
programs that provide both prevention and treatment services. A "club" approach, in 
which any youth in the school or community can apply or register with the program, 
means that youth and families are not stigmatized for joining the program. In fact, 
many of these programs emphasized during our site visits that they do not believe in 
identifying clients on the basis of presenting problems or dysfunction. In terms of 
designing an evaluation study, the multiple entiy points do not pose a problem as 
long as all entries and activities are properly recorded, as long as the program is able 
to distinguish clients from non-clients, and most importantly, as long as data from 
different types of clients are presented separately and clearly labeled. 



Program Services Offered 

Once the client enters the program, these programs use a variety of means to 
assess their service needs. Every program has some method for determining needs in 
a comprehensive way. Some programs use a relatively standardized assessment tool, 
comprised of one or more questionnaires or checklists typically developed elsewhere 
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and adapted for use by the program, while others rely on a more qualitative but 
thorough interview combined with a home visit to identify family environment issues. 
Many programs combine these two approaches. For the most part, the assessment is 
used to identify what services or program components the youth or family should 
receive while at the program, and also what other agencies may need to become 
involved. For some programs whose resources are not intended to be comprehensive, 
the assessment is also used as a screening tool to determine whether a given youth or 
family needs more than the program can offer and is therefore not appropriate for the 
program. 

All programs use some fonn of comprehensive needs assessment of individual 
clients (including families), but there are differences in whether they assess all clients 
or prospective clients. These differences appear to be related to the mix of prevention 
and treatment components offered. Those with a heavy emphasis on prevention (Teen 
Connections, Belafonte-Tacolcy, Houston Communities in Schools, Sunset Park, and 
Garfield Youth Services) usually do not conduct assessments on youth or families 
receiving prevention services unless a need arises. Conversely, clients who enter the 
treatment component of these programs always get a comprehensive assessment. 
Programs with more treatment-oriented services (Chins Up, Oasis Center) generally 
conduct assessments on all clients entering treatment. Exceptions to this pattern are 
I Have a Future and Big Brothers/Big Sisters. I Have a Future clients entering any of 
the program components— including the strictly prevention or enrichment 
components— must go through an assessment, including a required physical exam. At 
Big Brothers/ Big Sisters prospective clients and matches all receive an assessment. 

If programs required all new clients to take part in specific activities or program 
components, it would be relatively easy to identify entering youth or families. Among 



the prevention-oriented programs, only one, I Have a Future, explicitly requires clients 
to participate in specific program activities, although once clients complete these they 
are free to choose among a range of activities. None of the other programs offering 
preventive or enrichment services require participation in specific components, 
although many programs, by virtue of providing school-based workshops and 
curricula, have a relatively captive audience (Houston Communities in Schools, 
Belafonte-Tacolcy, Sunset Park). Others (e.g., Teen Connections) offer programs that 
all youth entering the program are encouraged to attend, but participation is not 
mandatory. In general, the treatment side of programs require more structured 
participation in a set of treatment-oriented program components. For example, at 
Chins Up residential shelter, youth must take part in a highly structured set of 
activities, including group discussion and workshops on health, sexuality, and 
communication. 

Due to the lack of explicitly mandatory participation requirements, it is often 
difficult to tell which youth receive preventive or enrichment activities on-site at a 
school, particularly when an entire class is the focus of the program. If a program 
such as Houston Communities in Schools provides on-site services at a school and 
students must register or sign up, then it is relatively easy to identify prevention 
clients. Even among programs that deliver some or all of their prevention activities 
within a classroom setting as part of the school curriculum, some consider the youth 
in the classes to be clients and some do not. Garfield Youth Services, for example, 
conducts a large part of its prevention program within classrooms, but does not 
consider the youth in the classroom to be clients unless the youth become involved in 
GYS outside of the classroom. Alternatively, Belafonte-Tacolcy conducts in-class 
workshops on drug abuse and gang prevention and does consider the youth in these 
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classes to be clients. At Sunset Park, youth who participate in the in-class arts 
enrichment curriculum are also considered clients, even if they do not officially 
register with the Sunset Park program. A program’s identification of youth in school 
classes as clients may be a function of both its need to show numbers served as part 
of a performance contract, and the nature of its contracted agreement with the school 
district. Given the variation in how these types < programs interact with schools, an 
evaluation should not expect consistent definitions of participants in classroom 
prevention activities as clients. 

Point oj Termination 

Finally, identifying which service recipients cire clients is made easier when the 
program has a clearly defined point of termination or departure. If the program 
maintains a case file that it closes upon termination, then we can say this individual • 
was a client. In addition, the definition that a program uses to consider a case closed 
is very useful in determining whether there is a clearcut point of termination. In the 
programs visited for this study, the point of termination was most clearly identified for 
the "caseload" clientele—those youth or families who received treatment-oriented 
services. At this end of the service continuum, most programs used termination 
procedures similar to those found in traditional services. At Chins Up and Oasis 
Center, youth leave the residential services program when they have spent the 
maximum allotted time in the shelter, when permanent placement or problem 
resolution has occurred, or when the youth has violated shelter rules (typically by 
violent or disruptive behavior, or by absconding). For youth or families receiving other 
services in these programs, the point of termination is reached in the traditional 
manner: either when treatment goals are reached or when the client refuses or resists 
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further treatment involvement. One prevention program, Big Brother/Big Sisters of 
Greater Miami uses clearcut termination procedures that are consistent with those 
carried out by more treatment- oriented programs. The big brother/big sister match 
will terminate if either the client or the volunteer decides to end it. Reasons for 
termination include: match goals being achieved, match incompatibility, or the client 
reaches 18 years of age, which is the oldest age for a client. 

Termination procedures for the prevention components within Belafonte- 
Tacolcy, I Have a Future, Sunset Park, and Houston Communities in Schools are 
more open-ended but follow a consistent pattern. For the most part, youth in these 
programs do not have to leave until they become too old to receive program services. 
Termination from specific, time-limited components within the program can be readily 
identified, even though clients do not typically leave the program as a whole. This 
holds true especially for Belafonte-Tacolcy, I Have a Future, and Sunset Park, where a 
"career path" for youth’s involvement with the program builds from initial prevention 
or enrichment activities. Youth in these programs can, and often do, stay in the 
program until they reach the maximum allowed age (18 for I Have a Future and 26 for 
Belafonte-Tacolcy). Active prevention cases may be closed when the youth drifts away 
from the program, attends irregularly or not at all, moves out of the community, or 
cannot be located. In all of these programs, extensive outreach efforts are made to 
locate the youth or family and to keep them involved. These include home visits, 
mailed letters, and accessing the family’s informal social networks in the community. 

For school-based programs such as Houston Communities in Schools, actual 
case termination occurs when the youth does not re-register for the CIS program the 
following school year. Usually this occurs because the family has moved or the youth 
has changed schools for family- related reasons. CIS staff will try to determine the 
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reason for loss of contact, but a relatively long period of time can pass before the 
program knows exactly why a youth or family did not return. 

In summary, clients in these programs may be identified by looking at how they 
came to the program, their pattern of involvement in program activities (particularly 
when programs mandate specific types of involvement), and how they end their 
contact with the program. Generally, it is relatively simple to tell who is a formal 
client in treatment components, since case files are kept for these clients. For 
prevention programs or the prevention component of mixed programs, the task of 
identifying which program participants to count is more complicated. Within any 
single program it is probably not too hard, working with the program, to decide who to 
include, how to do it, and what to measure. But variations in school and center- 
based program procedures and differences in length of program contact may make 
multi-site evaluations of prevention components a tricky undertaking. 

A final question raised by the findings of our site visits with respect to clear 
identification of program clients is whether the programs we selected may differ from 
other youth-serving programs in some systematic ways that affect the programs’ 
ability clearly to separate clients from non-clients. Clearly the service 
comprehensiveness these programs attempt and their frequent mix of prevention and 
treatment orientations means they attract more different types of youth, who have a 
choice among different ways to participate in the program. This feature of these 
programs alone suggests that telling the clients from the non-clients will pose a more 
complex challenge than in traditional single-focus programs. However, we have 
encountered exactly these same issues in examinations of many types of single-focus 
programs, so we do not believe they are unique to comprehensive or service 
integration program models. Nor do we think that programs focused on a narrowly- 
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defined segment of the youth population will experience any greater clarity of client 
definition, as long as the programs maintain some effort to meet client needs as 
identified. In fact, we think that only programs that limit themselves to a very specific 
treatment for a very specific time period are likely to achieve greater clarity about 
whom to count as a client, and even these programs will face the issue in an outcome 
evaluation of how to handle clients who drop out after minimal program contact. 

Client Risk Levels 

This section addresses whether the programs we visited have an operational 
definition of risk and whether they explicitly use their definition of risk to determine 
what services to offer their clients. Also of interest in this discussion is the degree to 
which the program’s working definition of risk conforms to the definition presented in 
Chapter 2 of this report. There, we suggested that youth be considered at high risk if 
they have at least one risk antecedent condition or risk marker and also display at 
least one risk behavior. Finally, we discuss the extent to which programs offer 
program components to youth based on their assessment of risk. 

Most programs use some type of risk definition but wide differences exist in the 
specificity of risk definitions. Further, different service components of the same 
program often use different definitions of risk. Programs that provide mainly 
treatment services, such as Chins Up and Oasis Center, tend to focus primarily on the 
youth and his/her presenting problems at intake. Risk is defined predominantly by 

9 

the presence of risk markers (i.e., out-of-home placement or school performance) and 
risk behaviors (i.e., drug use, juvenile delinquency, or family conflict), which is 
consistent with the empirical research on risk. However, entry into treatment is 
relatively independent of risk level and does not dictate what program components are 
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offered once the youth enters treatment. For example, Chins Up will not allow youth 
runaways Into the shelter unless they have had prior involvement with the courts, 
juvenile justice, or child welfare. For all program components except Detention 
Services for Juveniles (DSJ), levels of risk do not determine types of programs or 
services offered. In the DSJ program provided by Chins Up and operated out of the 
local detention center, a risk assessment score determines which of six levels of 
detention is recommended, from the most restrictive (secure detention) to the least 
restrictive (out on bond). At Oasis Center, the youth in the treatment component are 
not formally classified by risk factors, but a detailed intake interview and assessment 
provides information useful for identifying service needs and developing an action plan 
to meet those needs. 

In the more exclusively prevention oriented programs, such as I Have a Future, 
Big Brothers/ Big Sisters, and Teen Connections, risk is defined less according to 
already presenting problems or risk markers and more according to antecedent risk 
conditions, such as the neighborhood (in the case of I Have a Future) or the family 
environment (in the case of Big Brothers/ Big Sisters and Teen Connections). 

However, this approach means that the level of risk is equal for all program 
participants and does not determine the receipt of specific services. The prevention 
activities in Teen Connections, Big Brothers/ Big Sisters, and I Have a Future are open 
to almost any youth who meets the entiy criteria. But these programs do identify 
different risk levels through their assessment processes, which may lead to the offer of 
additional services. In addition. Teen Connections has learned over the years that it 
has to screen out very high risk teens because the program is not equipped to handle 
these youth. 

Programs that offer both prevention and treatment components (most of those 



in this study) use their risk assessment to identify high risk youth. Youth considered 
high risk are offered treatment services while all others are offered the preventive or 
enrichment programs. The approach may best be described as a form of triage. Little 
distinction is made in these programs between low and moderate risk youth, perhaps 
because most programs do not serve truly low-risk clients. 

Youth are deemed eligible to receive the most basic prevention programs if they 
meet the criteria for low or moderate risk. For most programs, this means that the 
youth live in specific antecedent conditions such as poverty, single parent households, 
neighborhood drug use, or violence. Further, they may or may not display some of 
the common risk markers such as poor school performance or risk for out-of-home 
placement. These minimal criteria qualify youth for the prevention programs. 
However, these mixed-approach programs do not keep relatively low-risk youth from 
participating in prevention activities. High risk is determined primarily by whether 
the youth exhibits any risk behaviors or risk markers such as those identified in our 
model in Chapter 2. Most programs consider the following risk behaviors in their 
assessment of high risk status: violence, truancy from school, substance or alcohol 
abuse, and police or juvenile court involvement. 

The pattern that emerges from the programs, particularly those providing both 
treatment and prevention activities, is that they use risk assessment information to 
make a decision regarding where to place youth in the service continuum. While this 
is a general pattern across all programs, some differences exist in what kind and 
amount of risk assessment information is actually used to assign services to clientele. 

Program managers at The Belafonte-Tacolcy Center consider all youth from 
Liberty City to be at risk. They then identify high-risk youth on the basis of risk 
markers--particularly poor school performance— and risk behaviors such as drug use, 
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gang membership, and school dropout. Youth with any of these risk markers or 
behaviors are eligible for counseling. The Center for Family Life in Sunset Park also 
considers all families living in the Sunset Park community to be at risk and assigns 
higher risk to those families where there are existing crises. High risk families include 
those in which the youth show the specific risk marker of high risk of out-of-home 
placement. Houston Communities in Schools considers all youth in the school to be 
at risk, since the program deliberately selects schools where most or all youth are at 
risk. It then identifies youth at higher risk on the basis of either risk markers, 
particularly school performance problems, or risk behavior such as drug abuse, 
conflict with parents, truancy, or gang membership. In all these programs, youth at 
lower risk receive the recreational and enrichment programs, while those considered 
at high risk also become involved in the counseling and remedial activities. 

At Garfield Youth Services a different pattern prevails. Some youth— those at 
high risk as indicated by their involvement with public agencies— are referred directly 
to treatment (counseling or case management). As part of intake they complete a 
questionnaire covering the areas of family, substance abuse, self-esteem, mental 
health, behavior, life skills, and peers. The program scores this intake questionnaire 
and uses the results to determine what issues to cover in counseling; it re-administers 
the questionnaire to assess progress at the end of counseling. Other youth enter the 
program through the prevention components. Those in one prevention component— 
the PALs mentoring program— also go through an assessment of risk. Those 
considered at higher risk go to the head of the waiting list for a PALs match. If risk 
appears high enough, they may also be referred for counseling. 



Youth/Family /Neighborhood Orientation 

While all of the programs visited for this project ultimately serve at-risk youth, 
their efforts may be focused at the individual, family, or community level. A program’s 
mission statement and major objectives can be used to identify its primary 
orientation. Knowing whether a program is oriented toward youth, youth and 
families, families primarily, or the entire community helps us to understand 
differences between programs in the process and content of service delivery, and this 
knowledge is vital to planning cross-program evaluations. There may also be 
differences in orientation even within a single program that ultimately influence a 
program’s goals or services and their achievement/outcomes. 

This section describes the programs’ primary orientations and how these 
orientations translate into different services. While most programs tend to orient their 
services toward one group, they often provide some services and programs to others. 
The boundaries between a program’s primary and secondary orientations are fairly 
explicit in programs that focus primarily on youth or community. Distinguishing 
between primary and ancillary orientations becomes less clear in programs that 
provide services to multiple groups, especially parents and families. Overall, the 
programs can be classified as follows: 

• One program focuses its efforts almost exclusively on serving youth (Teen 
Connections). 

• Five programs focus their efforts primarily on the youth (Oasis Center, Chins 
Up, Big Brothers/Big Sisters, Houston’s Communities in Schools, and 
Belafonte-Tacolcy), although these programs also involve families in program 
activities to varying degrees. 

• Three programs have broader orientations (I Have a Future, Center for Family 
Life in Sunset Park, and Garfield Youth Services). They are oriented strongly 
toward serving youth, their families, and the surrounding community. In fact, 
GYS expends considerable effort to maintain a public image as serving the 
entire community. 



Teen Connections is a youth-centered program and all of its activities are 
strongly youth-focused. There is little effort to provide services to parents or families. 
Its Teens for Teens component encourages teenagers to develop and practice 
leadership skills and serve as peer leaders to younger participants as they design and 
implement community health projects. While serving youth is its primary focus, Teen 
Connections does have a community action component that will be discussed later. 

Several programs focus their efforts primarily on serving the targeted youth, 
and services to parents and families are usually ancillary. Programs that focus on the 
parent or immediate family as a means of serving the child typically offer services 
such as parenting skills classes to help parents improve their understanding and cope 
better with their child. Oasis Center offers parenting programs at the Center and at 
parents’ work settings. Additionally, Oasis Center provides family members with 
crisis counseling or longer term counseling to deal with family issues that surface 
during the course of serving the youth, if needed. Garfield Youth Services has similar 
programs, but sees them as a major program component rather than as ancillary. 

The overall philosophy of the other programs requires that others who are 
significant in the lives of youth become involved in the treatment or services to the 
youth who are the primary clients. CIS takes a holistic approach to service delivery 
that focuses attention on families and family issues (such as hunger, housing, 
employment, literacy) as well as the youth, and the community role in identifying the 
needs to be addressed at their local CIS site. Big Brothers/ Big Sisters mandates 
parental involvement in a child’s overall treatment as a precondition to their child’s 
participation. They also provide referrals to families in need of additional services. 
Belafonte-Tacolcy is youth-oriented but will involve parents and other immediate 
family as needed, usually in treatment. This program has adopted a comprehensive 



developmental approach to serving youth; it has the capacity to engage youth from 
pre-school to their mid-20s by offering programs geared toward different age groups. 

All of Chins Up’s program components focus on the youth, and the level of 
familial involvement varies depending upon the program. Chins-Up helps to illustrate 
intra-program differences in client orientation. For instance, its Family Preservation 
Program component focuses heavily on the whole family as a target rather than the 
youth, although it is the youth who has the "initiating problem." Chins-Up’s Family 
Therapy targets the entire family; its Therapeutic Foster Care program targets the 
youth first, followed closely by the foster family; and its Residential Shelter and 
Detention Services for Juveniles targets only the youth. 

Three of the nine programs have a strong neighborhood o immunity 
orientation, with varying degrees of emphasis on serving individual families or youth. 
The Center for Family Life in Sunset Park has a strong community development 
philosophy. Its overall goals are capacity building within the community and 
empowerment of the families living there. This program is strongly oriented toward 
serving families and, unlike some of the programs discussed previously, does not view 
the family as simply a vehicle with which to influence at-risk youth. Garfield Youth 
Services has a very heavy focus on parental involvement and also on the community 
as a whole. Parents are virtually always involved in the counseling and case 
management components of the program. GYS counselors strive to identify the needs 
and capacity of parents and children, and to play a mediating role to strengthen 
communication among family members. Much of GYS activity is also focused on the 
community. Program representatives consistently stress their commitment to provide 
the services needed by the youth of the community and strive to draw parents and 
other community members into the process of providing those services and activities. 
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The I Have a Future program is explicitly neighborhood-oriented and focuses its 
efforts on obtaining participation from all of the community’s youth and families, and 
encouraging parents to become active in the community (e.g., by participating in 
tenant councils). This was not always the case, however. Originally, the program 
provided services through case management and brokering of services in the 
community. When the program altered its focus to on-site activities and a 
neighborhood orientation, its services and activities reflected this change. Their latch- 
key program is a clear attempt to devise programs to meet community needs. In this 
program, 6- 10-year-olds who are not program clients are supervised each afternoon at 
the program site by older "Future" participants. 

While not their primary focus, both Teen Connections and Belafonte-Tacolcy 
are involved somewhat in efforts to strengthen the community. The Tacolcy Economic 
Development Corporation is an offshoot of Belafonte-Tacolcy that raises money for the 
community and invests in the development of local properties (shopping malls and 
apartments for low income households). Teen Connections combines youth 
leadership with community development as youth design and participate in health- 
related community projects. 

Cultural Context 

Programs providing services to at-risk youth exist in a variety of settings and 
locales and may serve a culturally diverse mix of youth. It is important to understand 
the influence of cultural issues on the design, operation, and likely institutionalization 
of these programs in order to comprehensively evaluate a program’s success or 
impact. Overall, the programs in our sample serve clients from a range of racial and 
ethnic backgrounds. Some programs (I Have a Future, African-American; Belafonte- 



Tacolcy, African-American; Garfield Youth Services, white) primarily serve clients from 
one racial or ethnic group. Others have more of a mix, although the client population 
reflects a predominant racial or cultural group (Oasis Center, 80 percent white and 20 
percent African-American; Center for Family Life, 80 percent Hispanic with a growing 
Asian community; Teen Connections, 80 percent African-American, 20 percent 
Hispanic). Still other programs serve a diverse clientele (Big Brothers/Big Sisters, 50 
percent African-American, 25 percent Hispanic, 20 percent white; Chins Up, 60 
percent white, 14 percent African-American, 12 percent Hispanic, and others; and 
Communities in Schools, 51 percent African-American, 45 percent Hispanic, 4 percent 
white, and others). While Communities in Schools serves a diverse population 
throughout its 21 -school system, the clientele at given school sites tends to be 
homogenous. Big Brothers/Big Sisters faces intra-cultural issues that influence 
delivery of services, especially among their diverse Hispanic clientele. Additionally, 
one of our programs is located in a rural area. 

This section explores the impact of cultural issues on the following elements of 
our programs: 

• Program philosophy/ curriculum, 

•* Service delivery, 

• Staffing, and 

• Community perceptions /program ownership. 

Cultural issues affect multiple program elements in the majority of the sites. Exhibit 
6.1 illustrates the impact of cultural issues on program elements for each site. 

Program Philosophy 

A program's philosophy guides its goals and objectives and, ultimately, its 
structure and operation. The philosophy or long-range strategic plan of three of the 
nine programs reflected cultural considerations. The I Have a Future program has 
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adopted a program orientation that is culturally sensitive to its African-American 
clientele. By incorporating the Nguzo Saba Seven Principles of Blackness (unity, 
collective work and responsibility, purpose, self-determination, cooperative economics, 
creativity, and faith) into its curriculum modules, the program hopes to "...engender a 
more positive self-concept and constructive attitude toward community, family life, 
and the future...." 

Part of Oasis Center’s current strategic five year plan included the goal of 
"involving and serving minorities." The Center’s staff is almost exclusively white 
although approximately 20 percent of program participants are African-American. In 
line with the themes of "empowerment" and "opportunity" that are popular within the 
community, the Center plans to focus on multi-cultural diversity and to recruit a 
more culturally diverse staff. 

Garfield Youth Services maintains a philosophy of serving all county youth, and 
in that effort exerts itself to offer programming that will appeal to youth and their 
families, especially those who are not "in trouble." The program also sees its role as 
being a spokesperson for youth interests, and recently opposed selling beer at a 
county-wide event meant to attract youth, on the grounds that it set an example 
counter to the program’s drug and alcohol prevention message. 

Service Delivery 

Service delivery is the program element most frequently influenced by cultural 
considerations. Cultural issues have impacted upon the types of services offered and 
the process of delivering services in the majority of the program sites. In some 
instances, the needs of specific ethnic groups within a community have influenced the 
range of services provided by a program. To illustrate, Miami’s Belafonte-Tacolcy 
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Center collaborated with other agencies to create a comprehensive multi-service 
center to meet the needs of the community’s rapidly growing Haitian immigrant 
population. The Center provided workshops to deal with the concerns of new 
immigrants. ESL training, and assistance to parents who were registering their 
children in school for the first time. When the Center for Family Life in Sunset Park 
experienced an influx of Chinese immigrants, it provided ESL services and modified 
some of its programs to accommodate non-English, non-Spanish speaking 
participants. The Parent Council meetings currently take place in three languages 
including E gllsh, Spanish, and Chinese. Cultural issues also play a role in the 
service needs and issues identified by the different communities participating in 
Houston’s CIS program. Located in Houston’s "Little Mexico," the CIS program at 
Edison Middle School has addressed the lack of English language fluency among 
parents, illegal immigration, and ineligibility for the employment opportunities 
provided at the site. At Key Middle School, many of the CIS program activities focus 
on developing youth leadership groups and providing academic enrichment in 
response to a perceived lack of positive role models promoting academic achievement 
and discipline within the low-income, largely African-American community. 

Cultural awareness is also important during the course of service delivery. At 
Big Brothers /Big Sisters, social workers must be sensitive to potential conflicts 
inherent in inter-cultural matches. To illustrate. Big Brothers/Big Sisters program 
staff report that in Miami members of other Latin American groups tend to envy 
Cubans. Staff have found that this leads to lower rates of success for matches 
between a Cuban and an individual from another Latin American country. The 
program has found that matching non-African-American blacks (e.g., Haitians) with 
Hispanic participants is not typically an issue, but pairing an African-American 
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volunteer or child with an Hispanic volunteer or child tends to be problematic. In 
addition, program participants from the different Latin American countries may have 
different values and expectations that influence their participation in these types of 
programs. For instance. Nicaraguans are typified as private individuals who may be 
reluctant to disclose problems. This characteristic can affect a Nicaraguan family’s 
entrance into and progress within the program. 

Programs based in or providing services to rural settings (e.g., Garfield Youth 
Services) experience a different set of service delivery issues relevant to understanding 
the program’s approach and success. Most rural areas sorely lack the resources and 
services needed to assist at-risk youth. When GYS was created, for instance, there 
were no services available to at-risk youth in the catchment area and any services 
they provided met a need. However. GYS has found it important to be extremely 
sensitive to the language and values of each small community in the county it serves. 
This sensitivity extends to tailoring program advertising, references to the program, 
and content of services to the area involved. The benefit is that no part of the county 
feels neglected, and community members and public officials from every region of the 
county actively support the program. 

Staffing 

The diversity of a program’s clientele are also considered when programs make 
staff selections and specify staffing requirements. Those programs that serve a multi- 
cultural clientele have identified a need or have attempted to ensure that their staff 
reflects this diversity. Minority staff members not only bridge language barriers that 
might limit a client’s access to services (e.g, Chinese social workers for the Center for 
Family Life in Sunset Park) but may also serve as role models to minority clients. 
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Houston’s CIS staff reflects the racial and ethnic composition of its clients. The 
program also attempts to recruit "agency partners" and volunteers that reflect its 
client composition. The lack of African-American male volunteers at Big Brothers/Big 
Sisters has become a major focus of recent marketing and recruiting initiatives. The 
program spent over $30,000 several years ago to recruit African-American males 
through ads and flyers targeting minority-owned businesses, churches. African- 
American fraternities, and a popular African-American newspaper, but the return was 
negligible. The lack of African-American male volunteers prolongs the waiting period 
for African-American boys who wish to be matched with them. 

I Have a Future went through an interesting transition to ensure that their staff 
met the cultural needs of their clients. The project originally followed a brokering 
model, assessing needs and referring youth for services and recreational activities to 
agencies outside of the housing project in which the program was located for services 
and recreational activities. The project’s current executive director felt that these 
service providers did not fully comprehend the unique needs and complex situation of 
these African-American housing project residents. In reaction, she placed the services 
on-site in the form of curriculum modules on needed topics to be implemented by an 
almost exclusively African-American staff. 

Some programs favor multi-cultural staffing to highlight their cultural 
awareness and sensitivity. For example, both Big Brothers/Big Sisters and Belafonte- 
Tacolcy Center strive for a culturally diverse board that is knowledgeable about 
community Issues and needs. Chins Up makes sure that its residential shelter staff 
include African-American and Hispanic individuals to reflect the make-up of the youth 
residents. Conversely, Teen Connections severed ties with the national 
demonstration’s independent evaluator who was perceived to be insensitive to the 
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culture of Its program participants and upset youth with the types of questions asked. 
Of the four sites that make up the national Teen Connections demonstration, two had 
this reaction to this particular evaluator. 

Community Perceptions /Program Ownership 

Communities want to feel that their needs and interests will be met by the 
programs housed within their community. A community’s perception of local 
programs and feeling of ownership are important to its long-term viability. In our 
sample of programs, numerous factors influenced these perceptions and, ultimately, 
community ownership. One situation is a program whose board (or management) 
does not represent the community’s predominant racial, cultural, or socioeconomic 
status (e.g., Teen Connections). Communities may not have a sense of ownership in 
these programs if they perceive that their needs or interests will not be adequately 
represented. To facilitate community ownership, some programs hire past program 
participants and other community members as staff (e.g., Center for Family Life in 
Sunset Park, Belafonte-Tacolcy). Garfield Youth Services has taken a different 
approach to meeting the needs of the diverse communities within their rural 
catchment area. In order to ensure a program’s success, its programming and 
approach is tailored to accommodate character differences among these communities. 
These modifications may be as simple as altering program names in recognition of 
community sensitivities. 

A community’s perceptions may also directly affect program participation or 
success. Misperceptions about the racial, cultural, and socioeconomic background of 
program participants affect some Dade County community members’ willingness to 
participate as volunteers in Big Brothers/Big Sisters. A segment of older volunteers 
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for the Intergenerational Match program were hesitant to serve as mentors since they 
perceived that the youth the program serves are of low socioeconomic status, 'had" 
children from poor neighborhoods. The program applied marketing strategies to clear 
up this misconception. In Colorado Springs, the community’s perception that the 
area’s juvenile Justice system is racially-biased (in favor of whites) has made Chins 
TJp’s Detention Services for Juveniles program a risky venture. The community does 
not overwhelmingly support the program’s monitored release of Juvenile delinquents. 

As a result. Chins Up risks negative publicity and loss of community support if the 
released juveniles re-offend while waiting for trial on the earlier charge. 

Scope and Variety of Service Delivery 

The nine programs we visited for this project offer a very wide variety of services 
through their own auspices, and make an even greater variety of services available to 
their clients through either normal referral procedures or through special 
arrangements. In this section we describe the types and configurations of services 
offered by the programs themselves. This designation includes services offered at the 
program site by other agencies (co-location) and the few additional services for which 
some programs had made formal contractual arrangements (e.g., health care). The 
next section of this report, on service integration, discusses the service opportunities 
available to program participants through a number of additional arrangements that 
could be considered service integration. 

Before examining the specific services available at each program, we present an 
overview of the nine programs with respect to their location on a treatment-prevention 
continuum, and their location on a comprehensive-specific continuum. These 
characterizations are rough, but do provide a sense of this set of programs in relation 
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to each other. Once we examine service integration activities for these same 
programs, it will also become clear that no simple one-to-one relationship exists 
among these three domains: treatment-prevention: comprehensiveness; and service 
integration. 

Treatment- Prevention 

The nine programs we visited can be arrayed on a treatment-prevention 
continuum as follows: 

• Most treatment oriented: Chins Up, Oasis Center. 

• Mixed treatment and prevention: Garfield Youth Services, Center for Family 

Life, Communities in Schools. 

• Most prevention oriented: Belafonte-Tacolcy, Big Brothers/Big Sisters, I Have a 

Future, Teen Connections. 

Two of the nine programs concentrate almost entirely on the treatment end of 
the treatment-prevention spectrum: Chins Up and Oasis Center. These are also the 
two programs that run shelters for runaway and homeless youth and have a most 
explicit counseling/therapy /mental health orientation. Chins Up could be considered 
to do some prevention work through its Family Preservation Program, but families are 
referred to this program only at the point that a public agency is ready to remove at 
least one child from the home, so the program is not oriented toward primary 
prevention. Oasis Center offers small-group discussions/rap sessions in public 
schools which are designed to attract and assist troubled youth before they get into 
any major trouble. This part of the Oasis program is clearly preventive, but is not the 
program’s dominant activity. 

In the middle of the treatment-prevention spectrum are three programs that 
provide a balance of activities: Garfield Youth Services, Center for Family Life in 
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Sunset Park, and Houston’s Communities in Schools. These programs offer a blend of 
treatment and prevention, with about equal emphasis on each. Further, Garfield and 
Sunset Park are philosophically committed to involving the whole community in their 
activities and serving everyone, not just the youth and families in trouble. The 
consequence is that their program reflects their ongoing ability to respond to unmet 
need with additional program components cooperatively developed with other agencies 
and tailored to the community, whether these be treatment or prevention. Chins Up 
also shares this attitude and program development strategy, but concentrates on the 
treatment end of the spectrum. 

At the prevention end of the treatment-prevention spectrum are four of our 
nine programs: Belafonte-Tacolcy, Big Brothers/Big Sisters, I Have a Future, and 
Teen Connections. These programs try to reach youth before they become involved in 
negative behaviors. They offer alternatives of greater or lesser intensity--I Have a 
Future offers a place to be and a way to be, with other youth as supports, that 
actively counters the culture and negative opportunities of the housing projects in 
which the program is located. Belafonte-Tacolcy and Teen Connections offer their 
services both in schools and at a program site, with many of the youth reached in 
schools not able to attend the extended aspects of the program at its own site. 
Although these two programs aim to create an alternative environment with 
alternative norms and peers to support them, many of their clients do not participate 
in it. so the effect is weakened. Big Brothers/Big Sisters is a mentoring program 
without a formal location at which group activities take place. Its major focus is 
prevention (although case management services are offered), and its major mechanism 
for prevention is the persona), bond formed between a mentor and a mentee. Thus, 
even within the prevention-oriented programs we see variations on the degree to 



which the programs are trying to. and are able to. create alternate protective 
environments to counter the negative environments in which their clients live. Their 
success in creating an alternative environment reflects in part the strength of each 
program’s philosophical orientation toward its target-whether it is trying to affect a 
whole community, or trying to help individuals in a community. Among these four 
programs. I Have a Future is most articulate about affecting the whole community, 
and most developed in its efforts to do so. 

Comprehensiveness 

Here we examine the scope of concerns addressed by the program (e.g., health, 
education, drug/alcohol), whether in the prevention or the treatment domain. Almost 
all of these programs could arrange for a particular client to receive a particular 
service that is not offered by the program, if the need was pressing. But some 
programs specialize in addressing particular concerns, whereas others see their 
primary mission as affording access to a broad range of services and meeting a broad 
range of needs. Their assessment procedures will differ, as will the structure of their 
services and referral networks. We focus here on the programs’ main activities— those 
into which they put most of their effort and toward which their program structure is 
geared. 

Our nine programs can be arrayed on a specific-comprehensive continuum as 
follows: 

• Most specific: Teen Connections (health and pregnancy prevention): Garfield 
Youth Services and Belafonte-Tacolcy (drug and alcohol prevention, primarily): 
Oasis Center (mental health/ drug and alcohol); and Big Brothers/Big Sisters 
(mentoring). 

• Mixed: Chins Up. 
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• Most comprehensive: I Have a Future, Center for Family Life, and Communities 
in Schools. 

Five of our nine programs have a fairly defined focus, although the focuses 
differ. Teen Connections is narrowly centered on health and pregnancy prevention 
issues. Garfield Youth Services and Belafonte-Tacolcy center their activities around 
alcohol and drug issues, including both prevention efforts and direct service to youth 
and their families whose alcohol/drug-related behavior has already brought them to 
the attention of authorities. Belafonte-Tacolcy has branched out toward HIV 
education/ prevention, and therefore also into sexuality as well as substance abuse 
issues. GYS has taken on case management responsibilities for their most difficult 
treatment clients, many of whom have either personal or familial substance abuse 
issues and often also some court involvement. These extensions grow out of the 
programs’ primary focus, and do not really change the focus. 

The fourth program. Oasis Center, offers crisis intervention services to youth, 
with a mostly mental health/therapy/counseling focus and with a significant 
emphasis on drug and alcohol prevention. They have an on-site school (out-placed 
public school teachers) and a contractual arrangement for medical care because they 
operate a youth shelter and have youth in residence. Otherwise they offer no strictly 
educational or health services. All of these programs are able to connect youth 
and/or families with other services in the community on an as-needed basis, but do 
not have a major program emphasis on achieving comprehensive service delivery. 

Finally. Big Brothers/Big Sisters is predominantly a mentoring program. Case 
managers are available to connect youth to services as needed or noticed by the 
mentor, but the overwhelming program emphasis is on activities that the mentor and 
mentee undertake together, and on developing the relationship between the mentor 
and mentee. 
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Somewhere in the middle of the continuum is Chins Up. our ninth program. 
Chins Up’s services and case management for youth in shelter are very 
comprehensive. In addition. Chins Up as an organization takes on new programs as 
they are needed in the community, making it eclectic, flexible, and varied. Within the 
Chins Up Family Preservation Program, every conceivable type of service is made 
available to clients during 4-6 weeks of intensive casework, including cleaning 
apartments and developing a schedule for getting children to school on time in the 
morning. On the other hand, the detention alternatives program and the therapeutic 
foster care program are narrower in focus. Over all. Chins-up concentrates on 
treatment rather than on prevention. 

At the other end of the comprehensiveness continuum are three programs: I 
Have a Future. Center for Family Life in Sunset Park, and Houston’s Communities in 
Schools. The first two of these programs have an intense community focus, seeking to 
create a world within the larger environment that offers enough attractions to help 
clients withstand the stresses of the neighborhood and build toward the future. 
Whatever it takes to succeed at this goal is an appropriate activity for the program. 
Both involve youth, their families, and other community members. Both offer youth a 
broad range of activities, from sports and other recreation to tutoring to opportunities 
to assume responsibility within the program. Neither is particularly treatment- 
oriented in terms of serving youth. The Center for Family Life offers families 
assistance in accessing needed benefits and services: youth are involved in the 
Center’s programs as members of families, who are the primary clients. I Have a 
Future offers adults in the housing projects services directed toward co-dependency 
and chemical dependency: it does not concentrate on access to public benefits for 
families, but would help if a problem arose. 
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Houston’s Communities in Schools is organized quite differently from the first 
two comprehensive programs, but also covers a broad range of services and activities. 
Houston’s Communities in Schools operates on a co-location model; many agencies 
and services out-place a staff person at the school site, thus making services much 
more accessible to students and their parents. Services cover everything from drug 
and alcohol prevention to recreation to employment to mentoring to crisis counseling 
to juvenile justice. 

Specific Service Configurations 

Exhibit 6.2 summarizes the services offered by each of the nine programs we 
visited. This exhibit covers the services provided by the program itself, including 
those offered at the program site and those offered by program staff at other locations. 
It also covers services available to program clients through explicit contractual 
arrangements, such as the medical care contract that Oasis Center maintains with a 
local clinic to treat shelter clients when necessary. It does not include services that 
may be offered through routine referral procedures. These additional services are 
covered in the next section, on service integration. In Exhibit 6.2, a "Y” entry means 
the program offers the service to youth; a ”P" entry means that the program offers the 
service to the parent(s) of youth who are program clients; and a "C" entry means that 
community members other than the parents of program clients may take part in 
activities or services (and in the case of health services at I Have a Future, non- 
program youth who need school physicals may get them at Future’s clinic). 

The information in Exhibit 6.2 indicates that the most commonly offered 
services or activities among these nine programs are educational services (tutoring, 
school), mental health services (counseling, therapy, hotline), and recreation and 
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EXHIBIT 6.3: SERVICES PROVIDED BT PROGRAMS 1 
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NOTE: BT«Be lafonte -Tacolcy, Miami; BS-Big Brothers /Big Sisters, Miami; CU«Chins Up, Colorado Springs; GY«Garfleld 
Youth Services. Glenwood Springs. CO; Fu-I Have a Future. Nashville; Oa-Oasis Center, Nashville: SP« Center for Famiq 
Life in Sunset Park. Brooklyn; TC«Teen Connections, Bronx; ClS-Communides in Schools, Houston. 

1 Covers only services the program provides Itself, in any location, or services available to program clients through expllc 
contractual arrangements. Does not Include services that may be offered through referral. Y ■ offered to youth; P ■ 
offered to parents of youtn receiving services; C » offered to members of community, who may be parents, but not 
necessarily of youth served by program. 
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similar activities. The services least likely to be available directly from these programs 
are employment and training, help getting public benefits, and assistance in finding 
housing. Most of the programs are involved with criminal Justice agencies in various 
capacities, and also with social services agencies. 

Service Integration 

At the beginning of this project, we accepted a common set of criteria for 
defining service integration (SI), using five elements: 1) a view of the client from a 
holistic perspective and a commitment to provide services addressing a broad range of 
client needs: 2) a comprehensive needs assessment at client intake: 3) a service plan 
developed on the basis of the needs assessment: 4) a set of formal interagency 
arrangements designed to facilitate client access to services across programs: and 5) a 
record-keeping system capable of recording all service delivery, including that from 
referral agencies. Because we wanted to visit programs that were not already 
household words and had not already been the subject of extensive evaluation (such 
as The Door), we relaxed several of these criteria to include the final nine programs we 
visited. But we still went into the field with these criteria in mind. 

In hindsight, it is clear that the criteria we had in mind implied a particular 
program model. The clear implication of the five criteria is that a program has clients 
who need services (as opposed to activities or enrichment), that the services are 
available from some agencies other than the program, that various barriers make it 
difficult for program clients to get these other services from other agencies, and that 
interagency cooperation of several kinds will work to reduce those barriers and get the 
clients what they need. 

These criteria are all driven by client need. They all assume something 
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resembling a case management approach to helping people. They are not particularly 
relevant to programs whose major approach to helping youth is prevention- and 
activity-oriented. 

Even more important, the criteria ignore the potential SI relevance of program 
development. Several of the programs we visited do not appear to make extensive use 
of networks for referring their clients elsewhere. They do access these networks when 
appropriate, sometimes even using formal SI mechanisms such as multi-agency 
teams. However, these programs have made major investments in developing service 
components that other community agencies have identified as unmet needs. They are 
thus a referral source for government agencies, accepting as clients those youth whom 
government agencies do not have the resources to assist, filling in gaps in community 
services, and working cooperatively with networks of agencies to assure that 
appropriate resources are available. In some very real sense, they are the mortar that 
connects the whole community of youth-serving agencies. It seems important to us to 
include a discussion of these capacity-building efforts under the rubric of SI. Without 
such a discussion, the significant Investments of some programs in service 
development in response to community need could be overlooked, even though these 
efforts may be more important than formal SI in certain communities. 

Scope and Variety of SI Agencies and Arrangements 

Exhibit 6.3 provides a quick way to examine which types of agencies are 
included in the SI networks of the nine programs we visited, and the particular 
mechanisms through which they and the index program interact. The rows in Exhibit 
6.3 indicate the types of agency, system, or service that might be included in a service 
integration network for programs serving at-risk youth. These include the typical 
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service systems of education, mental health, housing, Income maintenance, health, 
social services, and criminal justice. Also included are activities not necessarily 
associated with a formal public service system, such as alcohol and drug prevention 
efforts, socialization/mentoring activities, and recreation/ fun activities. 

The columns of Exhibit 6.3 indicate the several mechanisms that may be used 
to provide services to clients through an SI network. Column 1 is not SI--this column 
indicates which services are provided on-site to program clients directly by program 
staff. Columns 2 and 3. inside the double vertical lines, indicate two methods of 
assuring service delivery that make up the core of what is usually thought of as 
service integration. Column 2 indicates that a service is made available to program 
clients through the mechanism of having staff from other agencies come to the 
program to deliver the service. These other-agency staff may be stationed at the 
program, as in Communities in Schools, or may come to the program on a regularly- 
scheduled basis, as do health care workers at the Chins Up shelter. In either case, 
these arrangements are formal and are undertaken for the express purpose of 
facilitating access to services and augmented service deliveiy. 

Column 3 represents the other common SI arrangement--a formal agreement or 
arrangement between the program and other community agencies to provide services 
for the program's clients. This may take the form of a contract for services, (e.g.. 

Oasis Center's arrangement with a downtown health clinic to handle the medical care 
of the Center's shelter residents). Or. it may take the form of a multi-agency team to 
which all relevant youth-serving agencies send representatives. The team meets 
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SERVICE INTEGRATION NETWORKS AND ARRANGEMENTS 
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regularly to handle the cases of clients who need services from several agencies. 
Several Chins Up programs and the Garfield Youth Services case management 
program use this type of team, and Sunset Park has a Human Services Cabinet which 
discusses cases relevant to several agencies on a less formal but still effective basis. 
By all accounts these arrangements greatly facilitate putting all the pieces in place to 
assist cases with rather complex needs. 

Column 4 depicts the situation in which the program has a contractual or 
other formal arrangement to provide services to clients of other programs on the site 
of the other program. The most common location of these other formal program 
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activities is schools, for which several programs have contracts or memoranda of 
understanding to provide teaching, enrichment, and/or casework services to school 
pupils. But one program operates a component in a juvenile detention center, and 
another has a staff person spend half-time at the offices of the juvenile court and 
DHS, the two agencies through which its shelter clients come. 

Column 5 indicates arrangements with other agencies that are frequent and 
relatively easy, but which are not based on a formal contract or memorandum. Big 
Brothers/Big Sisters is one of the best examples of this practice. Social workers with 
each of that program’s several components have developed their own network of 
referral agencies, with which they have built up substantial rapport and ability to 
obtain services for their clients. In addition to its formal links to DYS and DSS, Oasis 
Center also has many informal contacts as a consequence of knowing virtually all the 
agencies in town through years of joint work on task forces, committees, and projects. 
These informal arrangements are something more than normal referral procedures, 
which are not included in Exhibit 6.3. They are consciously worked out relationships 
with a personal element. In many instances caseworkers in the program and the 
referral agency know each other and have developed an understanding for working 
together. The program caseworkers have deliberately sought these relationships to 
improve their clients' chances of getting needed services. Insofar as they are more 
common than formal SI arrangements and they often work to facilitate service 
delivery, they should get some credit. But since they depend on personal 
relationships, the program’s ability to help clients access services breaks down if the 
relationship sours, or if one or the other party to it leaves. Many programs have 
found themselves having to rebuild their entire referral network from the ground up 
when a key staff person leaves. These types of informal relationships and 
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arrangements are no substitute in the long term for formal commitments. However, 
some informal arrangements (e.g., those between Oasis and a number of its referral 
agencies) involve agency heads rather than caseworkers. These arrangements tend to 
be more stable and consistent in the face of personnel changes than ones dependent 
on caseworkers. 

Finally, column 6 of Exhibit 6.3 shows the programs’ use of several types of 
volunteers to expand their service options. The most common volunteer activity is 
mentoring, but one program we visited provided emergency shelter through host 
homes with volunteer families, and both Communities in Schools and Belafonte- 
Tacolcy have numerous arrangements to provide services with volunteer members of 
the business community. 

The numerical entries in Exhibit 6.3 indicate how many of the nine programs 
we visited use a particular mechanism to provide services to their clients. Obviously 
the most common approach for many services is for the program staff to provide the 
service directly. Mental health, criminal Justice, and health agencies are the external 
agencies most likely to be involved in the formal SI arrangements depicted in columns 
2 and 3. either sending a representative to the program or to a multi-agency team. All 
but one of our nine programs could access mental health services through a formal 
mechanism, and all but two had formal ties to criminal justice agencies (usually the 
courts). While it may not seem to some that the courts should be considered a service 
agency, our programs sometimes could use the courts to require that another agency 
provide or pay for needed services (e.g., the Juvenile court Judge can tell DHS to pay 
for mental health care for a youth under DHS custody who is a client of one of our 
programs). The courts can also require a youth and his or her parents to attend 
counseling and other services as a condition of probation, so they may exert 
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considerable leverage over youth and even their families in particular aspects of 
program participation. The services least likely to be accessed through arrangements 
beyond the program were alcohol and drug prevention activities (in part because this 
was the specialty of many of these programs), recreation activities (again, a focus of 
many programs), and housing. 

Exhibit 6.3 also reveals that programs use the arrangements in columns 4, 5, 
and 6 less extensively than either performing the service themselves or operating 
through formal mechanisms. This finding is not likely to generalize to other youth- 
serving programs, however, since the nine programs we visited were selected because 
of their involvement in formal referral and service delivery mechanisms. 

Sharing Clients and Information 

In part because the formal SI arrangements used by the programs visited 
involve either co-location or multi-agency teams, the several agencies involved often 
serve the same client and usually need to share information. Most of the programs 
have worked out arrangements for release of information as needed, usually on a 
case-by-case basis. For instance, in the Garfield County multi-agency team, the 
agency bringing the youth for teaming obtains signed releases from the youth and 
parents to share information with the agencies that will attend. Once a plan is 
developed, additional releases are obtained to allow only those agencies with 
responsibilities to fulfill under the plan to share information. An agency that is a 
member of the multi-agency team but not part of a particular client’s plan will not 
know anything more about the client than what was shared at the meeting. The 
releases are only for the length of the plan; they are not general releases or permission 
to share information beyond the framework of the multi-agency team and the specific 



plan. Communities In Schools has an arrangement under which a parental consent 
form Is obtained for each youth participating in the program. This form spells out 
how information will be released to the various case teams. The consent form was 
developed to handle issues that had arisen with respect to confidentiality in the early 
days of the program. 

On the other hand, programs that do not have formal linkages but rather rely 
on informal arrangements did mention that information sharing was a continuing 
problem. Even when they are trying to get help for a specific client they have to talk 
in generalities. Another problem that arises with the less formal arrangements is that 
information that could be shared is not shared because there is no feedback 
mechanism to assure that a referring person ever hears what happened with a 
referral. An example among our nine programs is Belafonte-Tacolcy, in which 
participants in one program component may not know about other components in 
which they might be interested. Belafonte-Tacolcy has no in-house centralized data 
base to identify all the activities a given youth receives. Further, linkages in the 
schools with counselors and social workers often result in a youth being referred to 
Belafonte-Tacolcy for services without feedback to the school counselor about what 
happened. 

History of Service Integration 

The history and present status of service integration in each of the nine 
programs visited is detailed in the Individual program descriptions in Chapter 5. Only 
four of the programs -Chins Up, Garfield Youth Services, Center for Family Life, and 
Communities \.i Schools-participate in substantial formal service integration efforts. 
All were instrumental in developing the level of SI for youth currently found in their 
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communities. 



As described in more detail below, El Paso County’s Joint Initiatives project was 
the prime mover for the idea that eventually became Senate Bill 94 and multi-agency 
team efforts for both Colorado programs. Garfield Youth Services applied for and 
obtained one of these grants: GYS also has a long history of what we call "reverse" 
service integration, which we define and describe more extensively later in tnis 
section. Chins Up’s involvement in service integration began in 1989 (after Joint 
Initiatives began), with the hiring of the present executive director. This director is 
very outgoing i nd service oriented, willing to take on and develop needed services, and 
active in Joint Initiatives to identify what is needed and the best way to fill the need. 

All of the most innovative and "service-integrated" aspects of Chins Up programming 
are the result of his commitment: many are also the result of his involvement with 
Joint Initiatives. 

The Center for Family Life in Sunset Park has always had community 
development as a philosophical orientation and key goal. Staff saw service integration 
as the way to get their community development goals achieved. The program began 
with a more treatment-oriented approach, but during the past 8-10 years has taken 
on extensive prevention components that involve many elements of the community- 
both agencies and residents. The Center for Family Life was the major impetus in 
forming the Human Services Cabinet to bring all service providers together as part of 
the local Community District Board. Agencies belonging to the Cabinet see the Center 
for Family Life as playing a central role in how well the Cabinet functions: it is 
thought of as very well organized, doing a lot of outreach, and always available. It 
offers suggestions, shapes opinion, supports new developments, and generally eases 
the process toward more cooperation, more local opportunities, and more service expansion. 
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In Houston, Communities In Schools Is responsible for coordinating the 
services of different agencies that come Into the schools under CIS aegis. CIS goals 
were originally and remain comprehensive and Intentionally geared toward service 
Integration. These goals reflect the program’s nature as a local adaptation of the 
national Cities in Schools model with similar goals. Houston CIS has grown from one 
school and 1 1 cooperative agencies to 21 schools and a large and growing number of 
agencies and partners. One of the challenges for CIS is keeping track of the number 
of partners In each school and throughout the CIS system- -quite a task since each 
school develops its own community partners in addition to the formal government 
agencies that co-locat staff In many schools. 

Perceived Impacts or Benefits of SI 

The programs with SI arrangements in place and working well include Chins 
Up, Garfield Youth Services, Communities in Schools, Center for Family Life in Sunset 
Park, and one component of Oasis Center. Program staff members and 
representatives of other agencies in the community all credit these arrangements with 
helping them accomplish more, get more appropriate services to clients, and insure 
that each participating agency follows through on its commitments with greater speed 
and thoroughness. They say that SI also Insures that youth are much less likely to 
fall through the cracks. Another common perception is that the improved 
communication reduces the number of times that program staff ask an agency for 
something that the agency cannot do or in a way that the agency cannot handle. So 
SI has produced more appropriate requests for service, and requests framed and 
accompanied by documentation in a way that helps agencies to respond promptly and 
positively. 
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The two Colorado programs cite some concrete numbers to bolster their 
perceptions of the impact of SI. Garfield Youth Services has reduced the number of 
children and youth placed in detention (and therefore also out-of-county, since the 
county has no detention facility) by half, and has reduced the time in detention of the 
remaining youth by 77 percent. Chins Up and Joint Initiatives can also point to 
substantial reductions in out-of-county placement (30 percent the first year, now 
stable at 45 percent), in-county placement in foster care (through Family 
Preservation), improved foster care services (through Therapeutic Foster Care), and 
streamlined service delivery through several multi-agency teams. 

The Center for Family Life in Sunset Park adds another perceived benefit: more 
stable staffing patterns. Sunset Park staff feel that because of their community- 
building philosophy, staff get "hooked" on seeing what happens and making things get 
better. Their core staff has been with the program for a very long time. The schools 
also perceive that they get greater respect with the presence of Sunset Park programs 
in their buildings, and they report that youth do better in school and are happier as a 
result (no hard numbers are available). 

On the other hand, more than one program we visited expressed some 
reservations about formal arrangements, and did not have many formal linkages. 
These programs felt that formal arrangements reduced the program’s flexibility in 
finding Just the right service or agency for their clients. Thty were reluctant to 
commit to one or a few agencies because they did not '’/ant to have their options 
limited. 

Difficulties Encountered with SI and Approaches Taken 

The categorical funding streams so pervasive in programs and services for 
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youth are antithetical to comprehensive service deliveiy. Further, the rules and 
restrictions of these funding streams are the most important incentives to beginning 
SI efforts; if programs cannot change the rules, they use open communication and 
direct dealing across agencies to maximize the use of the services that are available, 
however awkward the process. We have not described in detail the difficulties these 
programs encountered in dealing with categorical funding because they have been so 
extensively documented in past assessments of SI. 

The most imaginative of the SI efforts we encountered— Joint Initiatives, in 
Colorado Springs-ultimately plans to incorporate and have all the dollars from 
different youth-serving agencies flow through the new corporation, with dollars 
following the youth, not stopping when a problem arises that cannot be handled by a 
particular type of categorical funding. All of the programs we visited try various ways 
to live with the disadvantages of categorical funding. Those with multi-agency teams 
or some other mechanism for facilitating communication among agencies with respect 
to particular clients appeared to do best at overcoming these disadvantages. The rest 
of this section describes difficulties other than those related to categorical funding and 
how programs have learned to handle them. 

Any change from the status quo, no matter how well-intentioned or needed, 
encounters difficulties as it is implemented. Many of the problems noted in past 
experience and research with SI were mentioned during our site visits, along with a 
variety of approaches taken to diffuse them. 

’Turf issues" were one of the most frequently mentioned difficulties. These can 
exist between agencies, between program staff and staff of an agency with which they 
want to work, and between ethnic groups. Sometimes these issues reflect a desire to 
keep control of a problem area, sometimes they reflect different expectations that 
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cooperating agencies might have for a program to which they all contribute, and 
sometimes they reflect different approaches to solving the same problem. Agency-to- 
agency issues mentioned during site visits included: 

• Several agencies competing for the same dollars to develop similar programs. 

• Agencies with control over some of the same youths (e.g., Juvenile court and 
DHS) falling to agree about the best approach, either for specific youth or for 
whole groups of youth with similar problems, and therefore not being willing to 
commit resources to the case(s). 

• Agencies with different goals for the program [(e.g., in Chins Up’s Detention 
Services for Juveniles, courts want youth to show up for hearing and not re- 
offend; the district attorney wants to know that community is safe; Division of 
Youth Services (which runs the detention center) wants fewer youth in the 
center (which operated at close to 300 percent capacity before DSJ began, and 
now runs at around 150 percent capacity)]. 



These are potentially serious issues which the programs have handled in 
several ways, all of which involve ongoing communication, patience, and creativity. In 
the Detention Center case, the program makes daily decisions that establish a 
precarious balance among the different agencies’ goals, keeps a close watch on the 
youth it releases, and hopes for the best. The Juvenile court/DHS conflict was 
resolved primarily by a change in personnel--a new chief Judge had a different attitude 
toward interagency cooperation and the two agencies and the program now have a 
good working relationship. The agencies involved in competitive struggles for money 
recognize that they have to change their approach once one agency gets a grant or 
contract for a new service. Their lines of communication are open enough that they 
cooperate with the new program so the community gets the services, but relations are 
sometimes touchy because issues of competition may surface at any time. 

Other agency-to-program and program-to-agency issues revealed during site 
visits include; 

• Disciplinary differences in approach that engender hostility or distmst (e.g., 





social workers from Sunset Park coming into schools to run an after-school 
program that teachers thought they should run. This problem had both turf 
and discipline aspects- -the social workers and the educators had different 
approaches to dealing with children). 

• Some key person in an agency feeling threatened by a program person with 
more extensive credentials or experience. 

• Program staff coming into a school at the busiest times of the school year and 
expecting school staff to help them; showing no flexibility or understanding of 
school procedures, schedules, or needs (in the perception of the school people). 

Ethnic tensions were also mentioned by one program. These were described as 
jealousy and contention over the distribution of resources, and whether agencies 
affiliated with and serving particular ethnic groups would get their own resources or 
would have to be under the control of agencies affiliated with different ethnic groups. 

In general, programs and agencies address these problems by pursuing the 
maximum level of openness and communication, often coupled with pragmatic help. 
For example, the Sunset Park social workers came into the schools during the school 
day. helped teachers with audio-visual equipment and special activities, and tried to 
win converts to the benefits to children of the after-school program. In addition, they 
added an academic (tutoring) component to the after-school program, to give teachers 
a role. These approaches worked to improve cooperation. Communities in Schools 
deals with person-to-person turf issues by developing memoranda of understanding 
that spell out the separate and complementary roles and responsibilities of the CIS 
staff person and the school personnel. 

Several programs mentioned the timing of efforts and the need for up-front 
negotiations to assure successful cooperative activities. Both Belafonte-Tacolcy and 
Communities in Schools noted the importance of having the school principal on board 
before bringing new services into a school. Belafonte-Tacolcy also noted that when 
seeking funding for new services, it was important to have the service sites in 



agreement before the funding came through. Their experience has been that these 
negotiations can take a long time, and may extend through a significant proportion of 
the grant period if they are not in place at the beginning. 

Belafonte-Tacolcy also mentioned another way that timing and money issues 
affected cooperation and service delivery. Sometimes cooperative arrangements have 
been developed for a specific project, which is funded for a limited period of time. 
Should that funding run out before new funding is found to replace it. the youth- 
serving program will have to reduce or eliminate the program, even if it may 
eventually seek to start it up again when new funding arrives. Agency staff find these 
ups and downs extremely disruptive and cooperative relationships may be lost for 
good if agency staff lose faith in the stability of program efforts. 

Finally, several different programs mentioned the effects of interactions within 
an agency on SI. Their comments indicate that successful SI and interagency 
cooperation depends on having the commitment of both agency directoi r and line 
workers. Programs that have tried SI with either one but not both have run into 
difficulties. If the SI effort is developed primarily at the line worker level without the 
support of the agency director, the line workers risk not being able to summon their 
agencies* resources when cases require them. Alternatively, if the SI effort starts at 
the top, no amount oi agency director commitment can produce better service delivery 
if the line workers either have not heard about it or do not like it. The agency 
directors we spoke to during visits to several programs voiced their recognition of 
having to do some training and reorientation with their line workers in order to make 
the new system work. 



"Reverse" Service Integration: GYS, Chins Up, and Sunset Park 

As noted at the beginning of this section, we began this study with an idea of SI 
common in the literature. Service integration is usually thought of as a way for a 
program to gain greater access for its clients to traditional services and benefits. 
Through formal arrangements with the agencies that control them, knowledge of these 
services and benefits, access to them, expedited eligibility determination, and speed of 
delivery are all expected to improve. This is a traditional model of SI--one that sees 
the impetus for SI as coming from within the program and seeking additional services 
outside the program. 

However, among our nine sites are three that serve a very different function. 
These three programs historically and routinely create services to fill the gaps in 
service identified either by other agencies or by the programs themselves. These 
efforts are cooperatively undertaken, with all the major agency players that might be 
part of a traditional SI network involved in their development. Once the new services 
are in place, some or all of these agencies refer youth or families with potential or 
actual problems to the new services. The program with the new services may or may 
not access the rest of the SI network for these new clients. As often as not, the 
program simply provides the new service or activity. Although not usually thought of 
as service integration, it seems to us that programs with this attitude and history are 
the very essence of a service system that truly meets the needs of its community. 

A Special Case: Joint Initiatives 

Joint Initiatives, in Colorado Springs, is a special case of service integration. 
Although it is not technically a youth-serving program, it fs classic SI at its best. 
Joint Initiatives (JI) is a formal collaboration of eight primary member agencies and 
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six associates with the goal of Improving services to children and youth in El Paso 
County. It began in 1988 with five agencies meeting to try to reduce the number of 
children and youth who had to be placed out-of-county. Within a year it formalized 
its operations. A primary member agency was defined as a government agency with 
official responsibilities for children and youth. Each primary member agency pays 
$10,000 a year into JI to maintain an office, executive director, and support staff. 
Primary members, each with four votes, are DSS, the county school district, Health, 
Mental Health. Youth Services (DYS). the district attorney, Juvenile court, and JTPA. 
Agency directors commit their agencies to be JI members, attend meetings, serve on 
committees, recommend specific projects, and have the power to commit their 
agency’s resources toward making any initiatives of JI work. 

Associate members are private agencies with a youth-serving mission. They 
pay $2,500 a year to be members of JI and are entitled to one vote. Associate 
members are Chins Up. Goodwill Industries. Head Start of El Paso County, the Myron 
Stratton Center (residential treatment facility for youth), and the social service 
commissioners from two adjacent counties. 

JI meets and identifies needs for service in the county. A committee is set up. 
usually with only 3-4 members. The committee meets frequently over a relatively 
short time period (2-3 months), develops a recommendation, and reports back to JI. 
Once JI approves the recommendation (possibly with modifications), JI staff take on 
the responsibility of finding funding for the project (so far. from state or other non- 
county funds, and/or state waivers to use county agency funds in new ways). With 
the money in hand and a program design on the table. JI puts the new idea out for 
bids to local service providers, some of whom may be associate JI members. Two 
Chins Up programs-Family Preservation and Detention Services for Juveniles (DSJ)- 
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were acquired through this process. In fact. JI’s interest in reducing the number of 
status offenders placed in juvenile detention led them to develop the idea for DSJ, 
selling it to the state in what became Senate Bill 94. and ultimately produced the ten 
demonstration projects of which Chins Up’s DSJ and Garfield Youth Service’s Case 
Management are two. Once contracted and running, each new service benefits from 
having a multi-agency team established specifically for the needs of its clients. The 
agency directors who participate in JI assure that their agency staff cooperate fully in 
these teams. 

Other JI projects on the horizon are out- stationing agency representatives in a 
multi- generational community center, developing a staff-secure residential facility 
(with security arrangements about half-way between the detention center and Chins 
Up’s current shelter, from which running away is easy), and developing an alternative 
non- residential school for youth in trouble. JI’s ultimate dream is to be able to pool 
all the children- and youth-serving funds coming in to each agency and use them in 
the most efficient and effective way (which everyone agrees is not the present way). 
They are contemplating various approaches to this, including incorporation and state 
waivers of numerous regulatory restrictions. 

JI is ambitious, cooperative, and very effective. Their realized projects have 
saved the county a great deal of money and resulted in better services for youth. It 
would be hard to find a more classic example of successful service integration that 
operates at both the top level of agency directors and the working level of caseworkers 
and program staff. 

Program Choices and Tradeoffs 

Programs continuously face choices of direction and focus as they grow and 



evolve. These choices shape a program’s focus, content and structure of service 
delivery, and ultimately, anticipated program outcomes. When conducting program 
evaluations it is important to understand how and why programs make the choices 
they do as well as each program’s ability to implement them. An awareness of key 
choices and turning points helps us to track a program’s development and evaluate 
the Impact and effectiveness of the choices on service delivery over time. 

This section describes the major choices and tradeoffs faced by the programs 
now and in the past. These choices are typically based on practical considerations, 
philosophical considerations, or both. The major decisions under consideration 
Involve: 

• Age range of the youth served. 

• Orientation toward youth, family, and/or neighborhood or community. 

• Orientation toward prevention or treatment. 

• Orientation toward providing services or activities. 

These decisions are seldom independent: most are interrelated. 

Age Range of Youth Served 

Programs interested in providing services to youth must decide who will be the 
target of service delivery efforts. Decisions to gear programming and services toward a 
particular age range of youth are not always clear cut. In many of the programs, this 
decision was integrally tied to a program’s focus on prevention versus treatment. In 
an effort to reach youth who will benefit most from preventive services and activities, 
programs such as Houston’s Communities in Schools arid Garfield Youth Services 
have expanded their targeted age range to include younger clients. In fact, as is the 
case with Garfield Youth Services, the younger the age of interest, the more preventive 
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is the mix of services and activities they offer. Services for older teens include both 
prevention and treatment. 

Houston’s Communities in Schools has responded to an Increase in the 
magnitude of problems experienced by younger children by increasing its presence in 
middle and elementary schools. By adopting a pattern in which youth participate in 
CIS in elementary, middle, and high school, they hope to intervene at an early stage in 
the child’s development and maintain contact with the youth for a longer period of 
time to prevent future difficulties. Garfield Youth Services also expanded the targeted 
age group for some of its preventive services, and Chins Up did so for its Family 
Preservation caseload. 

Decisions about the age range of clients has been a central issue for other 
programs as well. A key turning point in Oasis Center’s development was its decision 
to refocus its efforts from serving individuals of all ages to providing comprehensive 
services to meet the needs of teenagers. This decision influenced the types of 
programs and services offered to clients over the years and reflected both 
philosophical and pragmatic concerns. Oasis Center staff recognized the growing 
concern in Nashville over adolescent issues and felt that the program could 
proactively serve adolescents at the moment they were most likely to need help to 
avoid negative outcomes. At the same time, serving teens would let the program 
access and use available funding most effectively. 

The tradeoff evident when programs choose to target one age group is that they 
exclude youth of other ages who potentially need services. Some programs are 
structured to enable continued participation of clients over time. Belafonte-Tacolcy 
accommodates youth until the age of 26 and I Have a Future provides programs for 
youth between the ages of 10 to 17, plus a latchkey program for 6- 10-year-olds. 
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However, these programs do not offer some of the specialized services available in 
other programs (e.g., Oasis Center’s residential shelter). 

Orientation Toward Youth, Family, and/or Neighborhood or Community 

Earlier we described the primary orientations of the programs studied and how 
these orientations translate into services. Here we explore the reasons behind the 
programs’ choices. Each of the programs studied has chosen to focus its services and 
activities in very specific ways. Most often, decisions about focus are made early in 
the program’s life, although some programs have significantly redirected their efforts 
over the years. Program focus is evident in statements of mission, goals, and 
objectives: 

• "...to train teens to meet their own health needs...” (Teen Connections). 

• "...to provide coordination of services to at-risk youth and their families in a 
holistic way." (Houston’s Communities in Schools). 

• "...to develop a replicable community-based, life-enhancement program that 
promotes a significant reduction in the incidence of early pregnancy..." (I Have 
a Future). 

These program goals highlight the continuum on which a program’s orientation 
may be placed, from primarily youth-oriented (Teen Connections) to programs with a 
community or neighborhood focus (I Have a Future). 

The majority of the programs have chosen to tailor their services toward youth 
but have recognized the importance of including the family in some facet of service 
delivery. Some programs simply include family members in a child’s treatment plan; 
others view the treatment of a child holistically and direct services toward all facets of 
the child’s environment, including parents and family. Still others see the family as 
the focal point of its services (e.g., Chins Up’s Family Preservation Program or the 
Center for Family Life’s case management service). 



In some cases, a program’s goals or objectives may not be fully reflected In Its 
programming. Assuming the mission and goals are sound, exploring the lack of fit 
between a program’s goals/objectives and services is one way to evaluate the extent to 
which its goals are met. From its Inception, a key objective of Teen Connections’ case 
management component has been the involvement of significant others of the youth 
receiving services. This objective was reintroduced during year three of the 
demonstration by the program’s funders, who felt that the present service structure 
did not actively Include immediate family members in case management. 

One result of recent strategic planning at Oasis Center was the introduction of 
a new goal-advocating for youth and family service needs at all levels. Although the 
way in which this goal will be incorporated into programming is not yet clear, it may 
shift the program’s orientation from adolescents more strongly toward adolescents 
and their families. 

Orientation Toward Prevention/ Treatment or Services/ Activities 

Most of the programs’ services were either prevention- or treatment-oriented; 
this focus usually was related to the age range of the targeted client population. 

Other reasons for a program’s orientation may Include: characteristics of the targeted 
client population, the program’s mission, perceived need, or funding constraints. To 
illustrate, the original focus of Garfield Youth Services’ programming was drug and 
alcohol abuse prevention. They began to provide treatment services when local 
community agencies identified unmet service needs. The fact that Garfield Youth 
Services began to address these service gaps is consistent with their goal of serving 
the entire community. 

Once a program adopts a prevention or treatment orientation, or some 



combination of both, there is still some flexibility in the focus and comprehensiveness 
of its programming. Big Brothers/Big Sisters, for instance, has intentionally focused 
on prevention solely through its role modeling (mentoring) program. The tradeoff 
inherent in limiting its services to role modeling was the recognition that staff would 
need to build relationships with community agencies in order to ensure that they have 
a network of service providers to which they can refer needy clients. Houston’s 
Communities in Schools, on the other hand, provides a combination of prevention, 
enrichment, and treatment to clients at a single site, although the actual program 
offerings may differ by school site. 

A focus on prevention or treatment influences the types of services provided by 
programs. Prevention may include recreational or group activities geared toward 
building self-esteem (e.g., Belafonte-Tacolcy. Houston’s Communities in Schools), 
specialized groups (e.g.. Oasis Center), or structured curriculum modules (e.g., I Have 
a Future, Teen Connections). Treatment usually involves interaction with a case 
worker or social worker who facilitates a client’s acquisition of needed treatment (such 
as counseling, social services) or provides it directly. Prevention is typically 
introduced in the form of activities although it may include more structured services, 
while treatment is usually limited to the provision of specific services. 

A focus on prevention or treatment also influences the nature of and potential 
obstacles to evaluating a program. Prevention activities lend themselves to less 
rigorous evaluation than do treatment interventions. Case flies and records are 
typically maintained for clients receiving treatment services and these data can be 
used to assess the impact of treatment. In many of the programs, participants in 
prevention activities are not tracked regularly and specifications concerning who 
constitutes a client are less well defined. As a result, measuring the potential impact 
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of these interventions may be more difficult. Some prevention programs require their 
clientele to participate in a well-defined set of activities (e.g.. I Have a Future’s 
mandated curriculum modules). This makes it easier to evaluate client participation 
and other program impacts. To do prevention components Justice, evaluators would 
have to develop specific strategies appropriate for handling prevention activities, to be 
used in conjunction with more typical treatment-oriented evaluation methods. 



Evaluation Issues 

In this section we examine the programs’ readiness for an outcome evaluation. 
There are two key conditions that determine the readiness or "system" of a program 
for an outcome evaluation: willingness and capability. 

Willingness refers tc the attitudes, perceptions, and beliefs that staff 
communicate about the potential utility of an evaluation study and their motivation to 
help carry one out. In particular, we are interested in identifying any possible 
negative attitudes toward evaluation research as well as expectations for gaining 
benefits from an evaluation. Capability refers to the resources that programs already 
have in place that either enhance or constrain the potential for conducting an impact 
or outcome evaluation, including staffing patterns, resources, and informational 
systems. Finally, an important factor affecting both capability and willingness is the 
program’s history of participating in evaluation efforts, since this experience will 
contribute toward existing capabilities as well as the positive or negative staff and 
management attitudes about the experience. 

As might be expected, the programs we visited differed widely in terms of both 
willingness and capability. Generally, those with the highest levels of capability were 
also those with more positive attitudes toward evaluation, although there were some 
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cases where the two dimensions did not exactly correspond. First we examine the 
willingness of programs to conduct an outcome evaluation, considering their 
resources and the program’s readiness. This will be followed by a discussion of 
program’s capability to engage In this type of evaluation. 

Willingness 

Most of the programs show an interest In doing more evaluation research and 
assessing program outcomes in particular. Programs including Chins Up. Houston 
Communities in Schools. Garfield Youth Services. I Have a Future. Belafonte-Tacolcy. 
Big Brothers/Big Sisters, and Oasis Center express a high degree of interest in doing 
more evaluation. When expressing such interest, many Executive Directors 
specifically indicated that they want to do some form of longitudinal follow-up of their 
clients to see what happened years later as an indicator of the program’s success. 

The enthusiasm of some programs was related to earlier positive experiences with 
evaluation studies, as in the case of I Have a Future, which participated in a national 
research and demonstration project. The remaining programs may not have had 
experience with outcome evaluations, but they have had to report some types of data 
from their information systems and have conducted process evaluation studies. Some 
of these programs, such as Garfield Youth Services, are enthusiastic but somewhat 
naive about evaluation. The staff are highly motivated to do anything that would be 
required of them, but they feel they need technical assistance and so far have not 
been successful in finding any. These programs have not received any financial 
support to pursue development of an evaluation system, even though funders have 
suggested they do more evaluation. 

The programs that appear less uniformly positive toward evaluation research 
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are Teen Connections and the Center for Family Life in Sunset Park. Both are 
sincerely interested in evaluation but have had experiences with evaluators that left 
them feeling the evaluators were not sensitive to the concerns of their clients or were 
not able to reflect the complexity of client experiences in the program. Teen 
Connections has been working with an evaluator for the national demonstration 
program of which they are one of four programs. The evaluator imposed record* 
keeping forms that seemed inappropriate or offensive to the youth. The program also 
felt that the evaluator was not sensitive enough to the African-American youth 
population. The program is continuing to participate in the national evaluation, after 
making some adjustments in the way the evaluator deals with the program and its 
clients. 

The concern about evaluation at the Center for Family Life in Sunset Park 
stems from staff concerns that any evaluation not oversimplify o~ misrepresent the 
complexity of the program or of clients’ experiences in it. The program has had one 
experience with evaluation that staff felt was not an adequate reflection of the 
program- in particular they feel that the rather cut-and-dried approach to outcome 
evaluation used by one evaluation did not do Justice to either their services or the 
benefits their clients derive from the program. They did mention a professor from the 
Columbia University School of Social Work whom they felt would do a creditable 
evaluation, presumably because this person has the right training (direct social work 
practice) and also has done some work for them in the past. The Center for Family 
Life staff are also concerned that any evaluation deal adequately with data 
confidentiality and getting reimbursed for staff time spent on record-keeping and 
other evaluation-related duties. The Center for Family Life is now one of eight finalists 
to participate in a national evaluation of family support programs to be funded by the 



Annie E. Casey Foundation. Sister Mary Paul, director of Clinical Services, hopes 
their program is selected as one of the four programs to conduct the evaluation, and 
that they receive adequate guidance and support to conduct a high-quality study that 
reflects the Center’s wide range of clientele, many activities, and extensive community 
involvement. 

Capability 

The programs we visited can be grouped into low, moderate and high capability 
levels, based on their existing resources, research experience, qnd current level of 
documentation. The low-capability programs appear to lack the existing resources 
required for an evaluation study. Examples of these programs include Belafonte- 
Tacolcy, Center for Family Life in Sunset Park, and Teen Connections. In these 
programs, staff may not be knowledgeable about evaluation research and/or there are 
few concrete resources available to support an evaluation. In addition, information 
systems are at a relatively primitive level. While some information is collected through 
documentation of client backgrounds, monitoring of service utilization, and in some 
cases client goals, little of this is systematically aggregated, with the exception of the 
information required by funding sources. Information systems at these programs are 
not computerized and it is not always clear how information is aggregated for reports. 
There does not appear to be one central unit or department responsible for putting the 
information together. Such an effort would take only a few hours of a volunteer’s time 
every month to input the forms and construct the requisite reports. These duties 
appear to fall upon the shoulders of the top-level program managers such as the 
Executive Director, which means that if more important issues need attention, the 
record-keeping system does not remain current. For example, the Center for Family 



Life in Sunset Park uses a manual form of a spreadsheet on which the Project 
Director manually compiles the monthly statistics into a single year-end report. These 
programs also have some trouble tracking the involvement of outside agencies, which 
is an important component of documentation for service integration types of 
programs. For example, at Teen Connections and Belafonte-Tacolcy, once a referral is 
made to another agency or another agency provides services to a client there may not 
be much feedback or recording of the extent of the services received. 

We would place the Big Brothers/Big Sisters, Garfield Youth Services, and 
Oasis Center at the mid-level of evaluation capability. At a minimum, these programs 
maintain some form of computerized database system in which service and client 
statistics are input regularly. These systems are used to generate reports for funding 
sources and provide feedback to program staff on client flow rates, intakes and 
terminations, and client backgrounds. Typically, this capability also involves having 
staff whose job responsibilities include updating the database regularly by entering 
new forms as they are completed. These programs keep extensive records on what 
services clients receive, the length of stay of clients in the program, client 
backgrounds and assessment of risk, and information about the involvement of 
outside agency partners in either the referral or service provision processes. Some 
programs still rely on the Executive Director to analyze the service statistics (e.g.. Big 
Brothers/Big Sisters), but generally top management is supported by volunteers and 
staff who complete the forms and do the initial tabulation of the Information. 

The moderate-capability programs are also distinguished from the lower levels 
of capability by their ability to use the documented information for the purposes of 
planning and internal evaluation. For example, Garfield Youth Services has in place a 
system whereby risk assessment data are collected at two time intervals over a four to 
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six week period, at the beginning and end of short-term crisis counseling (for the 
treatment clients). Although this information is currently used only to inform 
counseling decisions, having the system in place gives GYS a higher capability for 
evaluation than some other programs. 

Another characteristic of the moderate-capability programs is that they have 
relatively well-formulated and sometimes quite specific plans for improving their 
evaluation capability. Oasis Center already collects client outcome data from some of 
its program components and it plans to incorporate outcome-based evaluation into 
others, including a ninety-day follow-up after clients complete the Early Intervention 
program. Big Brothers/Big Sisters has plans to expand their respondent base for the 
mail survey of matches by including past as well as current matches and by getting 
the views of the youth, the parent and the volunteer. 

Despite such well-articulated evaluation plans, this group of programs has 
some reservations about the potential tradeoff between the costs and benefits of an 
evaluation study. Big Brothers/Big Sisters is concerned that, since evaluation 
research activities are usually covered under agency overhead, they must be funded 
by indirect rather than direct service funds. Yet funders are reluctant to provide more 
money for administrative overhead, and as a non-profit agency, the program feels 
continually pressured to reduce overhead costs. Thus while the interest is high, there 
is a sense that the resources currently available for evaluation sire insufficient and 
that real obstacles exist to future evaluation research that would need to be overcome. 
Other programs in this group have articulated similar concerns although not as 
directly. 

Finally, programs with the highest levels of capability include I Have a Future. 
Chins Up. and Houston Communities in Schools. These programs have highly 



sophisticated, computerized management information systems and they have staff 
specifically assigned to do the data entiy. compilation, and generation of summary 
statistics to give top-level management current information ready for analysis. Hie 
most sophisticated level of documentation was done by the Houston Communities in 
Schools program, which has a Research Director and staff responsible for the design 
and operation of a relatively complete computerized management information system. 

A manual outlines all of the documentation forms and provides well-specified steps for 
completing the forms and sending them to the central office for data entry. All staff at 
schools are given a once-a-year training and update in the use of the information 
system and Project Managers at the schools are responsible for checking the accuracy 
of forms before they are sent to the central office. All programs have extensively 
documented all aspects of the program, including the prevention activities, and 
include data on their interactions with agency partners when these provide services to 
youth in the program. 

The high-capability programs have usually conducted some form of evaluation 
research in the past or are currently doing so. The evaluation studies have generally 
been small scale and primarily used internally to identify targets for planning and 
service utilization. The most established form of an evaluation system was evidenced 
by I Have a Future, which does pre-post assessments of a youth’s participation in a 
given program module, tracks service delivery over time, and has used a quasi- 
experimental design with a comparison group to assess outcomes. Similarly, Houston 
Communities in Schools was involved in an evaluation conducted by the University of 
Houston through funding from a local endowment. While Chins Up has not yet 
conducted this level of outcome evaluation, it appears to be ready to do so by virtue of 
how well it is able to track important service delivery, client participation, satisfaction, 
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and client outcome data. A wide range of information is collected from the youth. 

Chins Up staff, and agency partners who interact with the program such as the 
detention center, local mental health center, and health providers. Chins Up also has 
a variety of mechanisms already in place that would assist an outcome evaluation, 
including a one year follow-up of all youth in the shelter program, goal attainment 
scaling conducted by front-line staff, and a set of outcomes at termination from a 
program including the youth’s disposition when they leave the agency. 

An interesting finding among the high-capability programs is that they 
maintain excellent documentation and information systems as well as a high level of 
readiness despite some negative experiences with prior evaluation research. For 
example. Chins Up is part of a state-wide evaluation of state-funded programs offering 
alternatives to placing Juveniles in detention facilities. The Chins Up staff has 
substantial dissatisfaction with this evaluation: they perceive it as not meeting their 
needs, being overly intrusive and rigid, and not being competently performed 
(especially the forms they are given to complete, which appear lacking in sufficient 
operational definitions). The evaluation also suffers from conflicting expectations 
between the various stakeholders in the evaluation results. The Chins Up staff are 
concerned that the evaluation findings may negatively affect the program if the results 
do not match the overly high expectations of these stakeholders. Chins Up’s solution, 
which appears characteristic of all high-capability programs, involves implementing 
their own documentation and information system in order for them to maintain 
control over the site-specific data and how it may be used. 

All high-capability programs have high hopes for future evaluations and 
specifically want to track both prevention and treatment clients after they leave the 
program. Those programs located in other agency sites or involved multi-site services 
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also want to conduct a comparison between sites to identify planning issues specific 
to how the program is delivered at each site. This is particularly germane to the 
Houston Communities in Schools, since the program is currently operating in 21 
schools across three school districts. All of these programs clearly indicate that any 
costs accrued from doing evaluation research were more than compensated for by the 
benefits of the information obtained. 

Conclusions 

We can see that some of the programs we visited are currently ready to 
participate in an evaluation study while others would require additional resources, 
including an upgraded information and documentation system. A potential evaluation 
would also need to work with some programs to assure them about what an 
evaluation study can provide for them. If these programs also received added 
resources for such an evaluation they might view the enterprise more positively. 

Based 'Sn the above tripartite classification of program sites’ willingness and 
capability to conduct an evaluation study, we can make some general conclusions 
about the possibility of involving them in a multi-program evaluation. The high- 
capability programs appear the most ready to participate, and with some additional 
resources the moderate capability programs may also be helped toward evaluation 
readiness. Among this latter group, an evaluator would need to provide not only 
resources but also some additional training and technical assistance to ensure that 
staff on-site are capable of maintaining the evaluation effort over a considerable period 
of time. 

However, the real differences in program approaches and scope lead one to 
question the wisdom of including all of these programs in a single multi-program 



evaluation. Specifically, the differences between a prevention and treatment emphasis 
should be used to determine those sites most appropriate for inclusion in the same 
cross-site evaluation. For example, I Have a Future, Belafonte-Tacolcy, Houston 
Communities in Schools and Garfield Youth Services might be currently ready for 
such an evaluation. The Center for Family Life may be included in this group if the 
program gets some additional resources and technical assistance before the start-up 
period so the program can increase its systematic use of documentation and also gain 
assurance that the research design will adequately represent the program. A second 
group of programs appropriate for a cross-site evaluation may include Chins Up and 
Oasis Center, with possibly the addition of Garfield Youth Services (this program 
could be included in several configurations of cross-site evaluation plans given its 
diversity) and Big Brothers/Big Sisters. However, a third likely configuration of sites 
for an evaluation might involve Garfield Youth Services and Big Brothers/Big Sisters, 
since they both offer preventive, mentoring-based services. Finally, a cross-site 
evaluation might consist of all school-involved programs, including Houston 
Communities in Schools, Garfield Youth Services, Center for Family Life in Sunset 
Park, and Belafonte-Tacolcy. 

The only program that does not appear ready to participate in a cross-site 
evaluation is. paradoxically, a program that currently is involved in one of these 
evaluations-Teen Connections. Although it is part of an ongoing Kellogg Foundation- 
sponsored national research and demonstration program, it does not appear ready for 
evaluation. It has few resources available to track clients and services systematically, 
on-site staff levels are not adequate to support an evaluation, and the off-site 
evaluation group appears insensitive to the site-specific needs of this program. 
Finally, difficulties between Teen Connections and its host agency, the Girls Club of 
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New York, must be resolved in order to make this a viable long-term program. With 
so many obstacles, it is unlikely that an outcome evaluation will yield useful 
information. 

Across all programs visited, there are still a variety of issues that would need to 
be resolved before a rigorous evaluation study could be done. A major issue concerns 
the choice of comparison or control groups. It is not entirely clear from the site visits 
whether the programs would be able to identify a potential group of no-intervention 
clients. Although community-based controls may be formed, there are the risks of 
contamination with the program participants and the possibility of these youth and 
families entering the programs themselves at some point. One potential avenue for 
the choice of control or comparison group that should be explored is whether to draw 
these individuals from a matched sample living in an adjacent community. Most of 
the highly-capable and willing programs appear to have some impact on the overall 
community, so it may be interesting to compare effects between a community with 
such a program and a community without this type of program. Another important 
issue to resolve is the variability in information currently documented by programs, 
particularly service provision characteristics that would be amendable to 
classification. It would also be important to identify a standard minimum data set 
that all sites provide for the cross-site analysis. Additional measures specific to 
outcomes also need to be considered. All program sites should become involved in 
this decision-making process so they develop ownership and positive attitudes toward 
a cross-site evaluation. Given the special features of these programs, the measures 
should not just assess individual changes, but should also identify the effects of the 
programs on the community and on the inter-agency service delivery network. These 
are Just some of the key issues to address in order to design an outcome evaluation 





that includes some or all of the sites visited and that will identify the effects of 
comprehensive service provision and service integration models. 

Implications 

This chapter on cross-cutting program Issues makes clear the importance of 
visiting a variety of programs and examining how different programs approach the 
same issues. Only by making these comparisons can we see some important 
consistencies across programs and also gain an understanding of some program 
choices that may lead in distinctly different directions. One example of cross-program 
consistency is the role three programs play as the "community glue" or "mortar" filling 
in gaps in the service system. This role has been critical in creating the quality of 
service available in the communities reached by these programs; yet the actual service 
content and treatment-prevention focus of the three programs is very different. 
Programs that choose to emphasize prevention follow that choice with decisions to 
offer a particular mix of services and activities; their service mix is quite different from 
the service mix in heavily treatment- oriented programs. 

The sheer diversity of the program configurations we found in these nine 
programs attests to the creativity and determination with which program staff seek to 
meet the needs of their youthful clients. The records of success assembled by many 
of the programs suggests that their comprehensive approach, flexible attitudes, and in 
some instances long-term involvement with youth may be the keys to making a 
difference for high-risk young people. These programs* diversity combined with their 
demonstrated impacts also indicate that a wide variety of approaches can make a 
difference for youth. However, whichever approach is selected needs to attend to the 
complex life circumstances facing the youth and bring a consistent philosophy to bear 



on these circumstances. In this broader context. SI efforts designed to facilitate 
service delivery can help. 

Virtually all of the program directors with whom we spoke evinced a strong 
interest in evaluation, and In particular in outcome/impact evaluation. At the same 
time, one of their chief complaints is that many funders want to see documented 
results but no one wants to pay for the work involved. The program directors 
recognize that creating and maintaining a good evaluation system takes staff time and 
expertise; the staff involved would almost certainly be administrative, which then 
drives up overhead rates. Many funders who say they would like to see outcome data 
look askance at even moderate overhead rates, creating a Catch-22 for program 
directors. It is very clear that any plans for multi-program outcome evaluations will 
have to supply each program with adequate evaluation staff and resources and also 
provide overall guidance and technical assistance. 
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CHAPTER 7 



SUMMARY AND IMPLICATIONS 



Project Objectives 

We undertook this project to learn more about programs that address the 
needs of 10- to 15-year-olds at risk of engaging in negative behaviors and experiencing 
negative outcomes such as school failure, too-early childbearing, alcohol or drug 
abuse, or criminal involvement. The programs of specific interest are those that try to 
provide comprehensive coverage of many possible services and activities, and those 
that use service integration as at least one mechanism for increasing the 
comprehensiveness of their offerings. We also completed a summary of the research 
literature relevant to risk definitions and risk prevalence and ar. examination of 
evaluation issues raised by comprehensive programs using service integration with 
younger adolescents. 

Risk Definitions and Risk Prevalence 

After reviewing the variety of risk definitions present in the research literature, 
we proposed an integrative framework for thinking about risk. The framework 
includes four components: risk antecedents (poverty, neighborhood/ environment, 
family dysfunction), risk markers appearing in public system records (poor school 
performance, involvement with child protective services), problem behaviors (early 
sexual activity, truancy, running away from home, early us of tobacco, alcohol, or 
other drugs, associating with delinquent peers), and risk outcomes (e.g., pregnancy, 
homelessness, prostitution, delinquency, sexually transmitted diseases, other morbid 
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conditions, death by accident, suicide, or homicide). 

We then reviewed available evidence indicating the likelihood that youth aged 
10-15 would experience the antecedents, markers, behaviors, and outcomes. Efforts 
to develop estimates of prevalence of at least one of these elements among the youth 
population suggest that one-fourth of today’s youth aged 10-15 have a high risk of 
experiencing at least one antecedent, marker, behavior, or outcome, and another one- 
fourth run a moderate risk of having these experiences. 

Implications 

With one of every two youth today running at least a moderate risk, the most 
striking implication of our literature review is the sheer number of youth who could 
benefit from preventive and support services. We also conclude that many of the 
same antecedents and markers are present regardless of which negative behavior or 
outcome one examines. This is one of the most important reasons why the categorical 
approach to problem definition and program funding falls short of what many youth 
need. The "presenting problenT-drug use. or delinquency, or pregnancy-may be a 
proximate result of opportunity or chance, yet the solutions probably lie with efforts to 
address the causes underlying the problem behaviors. In our model those causes are 
represented by risk antecedents and risk markers. Addressing these causes often 
means accessing services well beyond the range of the services that any given 
categorical funding source will support. 

The model can also provide a guide to specify which data programs should 
collect to describe their clients life circumstances and experiences. Most programs 
can identify and could record their clients’ status with respect to the essential 
elements of this risk model. This information would be useful for many purposes, 
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including the development of individual service plans and the analysis of program 
impacts differentially by client risk level. 

Traditional Program* 

Traditional programs for youth usually focus on only one of the possible 
categories of behavior that can lead youth to negative behaviors. Thus we have 
programs to prevent delinquency, or programs to prevent or treat drug or alcohol 
abuse, or programs to prevent too-early pregnancy or to support teen mothers, or to 
address the mental health problems of youth. But too often these traditional 
programs do not help youth avoid risk and its outcomes because they focus too 
narrowly on a single type of behavior and cannot address the multiple problems and 
circumstances which may confront a youth. Traditional programs reflect the nature 
of funding for youth services; almost always funding is categorical, coming from 
criminal justice system agencies to address delinquency, from drug and alcohol 
agencies to address chemical dependency, from mental health agencies to address 
mental health problems, or from education agencies to address school failure. The 
mandates of the funding agencies do not permit them to go beyond their service 
domain, so the programs funded through these agencies usually cannot use the funds 
they have to serve all of a client’s needs. 

Implications 

Research indicates that narrow, single-focus programs have limited success in 
their objective of preventing negative outcomes or ameliorating the effects of negative 
behaviors. These research results, together with the frustrations of categorical 
funding experienced by programs, lay the groundwork for the search for alternative 



program designs. Programs have tried to extend their ability to serve clients in one or 
more of the following ways: by developing more comprehensive service packages 
themselves, by improving their access to services available in the community, by 
supporting the development of additional services by other agencies, and by 
reconfiguring service delivery to be more efficient and accessible. 

Service Integration Definition* and Issues 

The failures of traditional single-focus program approaches have led many 
agencies to try to increase the comprehensiveness of their approach and their service 
offerings. Many times an agency will develop and offer a new service itself as a way of 
making more varied and appropriate services available to its clients. Another 
approach, service integration, is often used in conjunction with service expansion 
within the agency, but may also occur by itself. Service integration, as it is usually 
defined, tries to make more services accessible to clients by increasing the 
coordination and cooperation among many agencies so the diversity of services 
available through the whole group of agencies are easier for the client to obtain. A 
primary goal of this project was to examine how service integration worked in 
programs for 10- 15-year-old clients. 

In this project we initially defined the type of program we wanted to look at as 
those meeting five criteria: "seeing the whole youth," including family and 
neighborhood influences and needs; conducting a comprehensive needs assessment at 
intake; developing a coordinated comprehensive service plan based on the 
assessment; maintaining and using formal interagency linkages to facilitate referrals 
and receipt of services; and follow-up on service referrals to assure that youth receive 
the services and that the coordination mechanism is working well. We also drew 
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distinctions between comprehensiveness, adequate levels of services available, and 
service integration. We noted that a program c :i aid be comprehensive within its own 
walls and not use service integration, and that it could use service integration but the 
integrating arrangements could fall short of providing access to comprehensive 
services. Further the absolute level of services available in the community could be so 
low that even efficient and effective service integration could fail to assure that clients 
would get needed services. 

Implications 

In past studies of service integration efforts, barriers to success were many. 
These included clashes between professionals trained in different disciplines, 
administrative procedures of different agencies, eligibility rules of different programs 
and funding sources, and the categorical nature of funding for most services for 
youth. Despite these difficulties, participants often reported that SI helped get 
services for clients and that fewer clients fell through the cracks as a result of service 
integration efforts. One implication of the results of past service integration research 
is that it is easier to modify the behaviors of institutions around the edges than it is to 
make wholesale changes in institutional structure and orientation. Thus multi- 
agency teams may be effective in connecting clients with services because team 
members know how to work their respective systems: but the systems themselves 
have not changed much or at all. For the systems to change, funding streams would 
have to change, and few hold out much hope for a major advance in that area. 
Nevertheless, benefits to clients do result from SI efforts involving multi-agency teams, 
outplaced agency workers, and other streamlining mechanisms. 
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Evaluation Issue* 



In Chapter 3 we looked at nine evaluation issues pertinent to service 
integration programs for at-risk youth aged 10-15. These included defining the client 
and the unit of analysis; the impact of client risk levels on selection into programs, 
service receipt while in programs, and analysis of outcome data; documentation of 
service delivery; non-client outcomes of interest; documenting comprehensiveness and 
service integration and differentiating the impacts of each; evaluation willingness and 
capability among programs; identifying realistic outcomes to measure; identifying 
appropriate comparison groups; and reducing attrition at follow-up. 

Implications 

No perfect evaluation design exists that will fit every program, but in general we 
draw several conclusions from our review of these issues. We suggest that program 
evaluations use point of program entry as the time from which follow-up is calculated, 
and that evaluation designs be adapted and expanded to accommodate the variety of 
ways that youth, parents, and community members can become involved with the 
program. We think it is essential to record factors that indicate client risk levels, and 
to use these data as part of any analysis of program impact to prevent 
misinterpretation of results. The best approach appears to te to analyze results 
separately for client groups at very different levels of risk. For meaningful evaluations 
of these complex programs, it is important to document all service delivery so the 
evaluation can examine the program as delivered to each client, which may vary 
considerably from client to client. Many evaluations will want to include 
documentation of non-client outcomes such as changes in agency procedures, speed 
of completed referrals, services delivered that would not have been accessed without 



service integration mechanisms, money saved, or new services developed. Outcomes 
measured should be appropriate to the program goals, and should reflect the full 
scope of what the program is trying to accomplish. Multi-program evaluations should 
take care not to reduce outcomes measured to the few that all programs share but 
that reflect only a small proportion of the effort of any program. If at all possible, 
evaluation designs should include comparison groups, as they greatly strengthen the 
credibility of any results obtained. Finally, every effort should be made to assure high 
rates of completion of follow-up data collection from both treatment and comparison 
groups. Low completion rates compromise conclusions drawn from any evaluation. 

Site Visit Objectives and Procedures 

We conducted visits to nine youth-serving programs in six locations. We 
deliberately selected diverse sites; they represent a broad range of program 
configurations, clientele, service goals, and orientation toward prevention or 
treatment. However, all were selected because they meet most or all of the criteria for 
service integration and try to offer a comprehensive array of services to their clients 
and community. These programs are not typical of all youth-serving programs, but do 
represent different approaches to serving the whole child and his or her family and 
community. 

The nine programs visited came from a much larger list of programs generated 
by asking many experts familiar with youth-serving agencies to recommend programs 
for us to visit. We narrowed the list by calling programs to verify the nature of their 
service configuration and clientele. The final selection was made to balance 
geographical location, client characteristics, program orientation toward prevention or 
treatment, and school or community base. Two project staff visited each program. 
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interviewed staff, clients, parents, board members, and sometimes other community 
representatives, and examined recordkeeping systems and program reports. 

Characteristics of Sites Visited 

At least half of the clients served by these nine programs are 10-15 years of 
age: for some programs the proportion is close to 100 percent. One program serves 
only girls: the remainder serve both boys and girls, but have more boys the more they 
are connected to criminal Justice system agencies. Two of the programs serve almost 
entirely African-American youth, two serve mostly white youth, one serves mostly 
Hispanics. two serve a mixed Hispanic /African-American group of clients, and two 
have very ethnically diverse groups of clients. 

Three programs focus their efforts mostly or exclusively on the youth 
themselves, but may assist a youth’s family if it becomes apparent that help is 
needed: three programs focus on youth in some of their activities and place a heavy 
emphasis on involving the families of youth in other components of the program (e.g., 
for "caseload" clients): and three programs have some activities mainly for youth, 
some services that involve youth and their families, some offerings for any interested 
community member, and an overarching goal of changing and empowering the whole 
community. The nine programs include one mentoring program, one program 
focusing on a particular geographically defined community, one operating almost 
entirely in the schools, three operating in both schools and the community, and three 
that are community-based. Five of the programs use case management, and three of 
the programs offer crisis-oriented short-term services. 

Most of the nine programs have long histories in their communities. Two have 
operated for more than 20 years, and five began between 10 and 15 years ago. The 
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two newest programs both began as demonstrations, still rely on demonstration 
funding, and are less than five years old. Five of the programs visited meet all five of 
the criteria established for program selection. One program meets four criteria, two 
programs meet three criteria, and one program meets only two of the criteria. The 
criterion most often missing is the ability of programs to obtain and record successful 
service delivery by agencies to which they have referred their clients. Also, several 
programs do not have formal interagency linkages to facilitate service integration 
(although they do have informal arrangements and understandings with other 
agencies). Chapter 5 described each program in terms of a brief history; its current 
mission, goals, and objectives; its service configuration; its current clientele or users; 
the type and makeup of its SI network; its funding sources; and its experience with, 
interest in, and capability for evaluation. 

Implications 

The nine programs visited during this research were selected deliberately for 
their comprehensiveness and inclination to participate in service integration efforts. 
Selection was further balanced for the agency’s substantive focus and program model. 
Thus these nine programs definitely do not represent youth-serving programs in 
general, and should be taken as illustrative rather than representative of 
comprehensive youth-serving programs involved in service integration. Further, they 
definitely do not represent all youth-serving programs, most of which continue to 
pursue a single-focus model. 
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Cross-Cutting Issues 

Chapter 6 presented the conclusions from our site visits with respect to a 
number of cross-program issues. 

Who Is a Client? 

We found that the treatment components of programs have the clearest 
procedures for identifying clients and determining their point of program entry. Many 
prevention components also can identify clients and their point of entry because the 
programs have formal intake procedures for these components. But prevention 
components that involve public presentations (e.g., at schools and in other community 
settings) usually record only numbers of people attending. 

Implications . Evaluations of programs that have the level of complexity found 
in these nine programs should take pains to create an evaluation design that does 
Justice to that complexity. This means: 

• Using point of program entry as the starting point for longitudinal data 
collection including follow-up for caseload clients. 

• Providing some activity/service codes that are unique to each program, as well 
as those that all programs have in common. 

• Developing supplementary mechanisms for recording purely prevention efforts 
(where no "intake" happens and no individual client records are created or 
maintained). 

Client Risk Levels 

Many programs document client characteristics and behaviors that could form 
the basis for determining risk level. Some of these programs also use client risk level 
information to determine what services or program components to offer to clients. At 
least one program uses risk level information to screen out applicants whose needs 
exceed the agency’s capacity to handle. However, since most of these programs do not 



do outcome evaluation, risk levels do not figure in any interpretations of program 
impact. Even the programs that do assess some outcomes do not structure their 
evaluation efforts to include a consideration of how services received or initial client 
risk level affects outcomes. 

Implications . Client risk levels do not currently play a major role in program 
data collection or evaluation. More explicit thinking about the actual and potential 
uses of client risk levels in youth selection into a program, assignment of services 
once in a program, and program impact might contribute to revised program design 
and data collection. Anyone contemplating an evaluation- -whether of a single site or 
of several sites— of youth-serving programs such as those visited during this project 
must pay explicit attention to client risk levels if evaluation results are to be 
meaningfully analyzed and interpreted. 

Program Focus and Service Configuration 

The types of services and activities provided by these programs vary widely. 
Programs differ as to whether they focus heavily on prevention activities or heavily on 
treatment, or whether they offer a mix of prevention and treatment components. They 
also differ on whether the services and activities offered span many different 
substantive areas (e.g., drug and alcohol abuse prevention, mental health, pregnancy 
prevention and health promotion, delinquency prevention) or concentrate fairly 
narrowly within one or two areas. Yet a third way they differ is internal— youth in 
some program components may have access to a very comprehensive array of services 
but youth in other components may not. Usually the youth offered a more narrow 
array (typically prevention/ enrichment) are not considered to need the more intense 
treatment options, but can get them if the need becomes apparent. 



Programs appear to have made choices among an interrelated set of Issues In 
designing their programs and structuring their growth and change. Programs that 
gear their activities more toward prevention elect to serve the younger end of the 
youth age spectrum, their service mix reflects more activities and enrichment 
compared to formal services or treatment, and they tend to involve family and 
community orientations in addition to providing activities Just for youth. Most of the 
youth they serve tend to be in less immediate trouble; their risk level is defined by 
their families’ poverty, single parenthood, and/or residence in a neighborhood 
characterized by poverty, crime, and high rates of chemical dependency and non- 
marital births rather than by their own behavior. The opposite choices with respect to 
most of these dimensions characterize the programs with a stronger service/treatment 
orientation. Most programs have reconsidered their balance on these dimensions one 
or more times over the years and have added to their service array or shifted emphasis 
depending on the policy decisions made during these periods of review. In general, 
however, programs that began as prevention have maintained prevention as their 
major emphasis and programs that began as treatment have also maintained this 
emphasis. 

Implications . Because decisions about which youth to target and how to 
structure a program are so interrelated, researchers would have a difficult time trying 
to separate from one another the effects of several of the program dimensions 
described in this report. Further, the mix of services, orientations (prevention- 
treatment), and coverage (comprehensive-focused) varies so much from program to 
program that the task of designing an appropriate multi-program evaluation becomes 
extremely difficult. A great deal of negotiation with each program would be necessaty 
to assure that an evaluation design includes critical aspects of each program’s 
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offerings while still having enough in common across programs to justify a combined 
evaluation approach. But the effort to create a multi-program design is likely to be 
worth the work if it results in a clear reading on what parts of program structure and 
delivery make a critical difference for youth outcomes. 

Service Integration 

Cutting across all of these programs’ different patterns of service delivery is 
their involvement with formal service integration efforts. The site visits confirmed our 
initial view concerning the variety of program configurations used to facilitate access 
to services and augment service delivery to program clients. Programs use both 
formal and informal arrangements. In the formal category some programs: have staff 
from other agencies come to deliver a service, either on a permanent or a scheduled 
basis; contract with other community agencies to provide services or participate in 
multi-agency teams that meet regularly to handle clients needing services from several 
agencies; and have contracts to provide services to clients of other programs on the 
site of the other program. In the informal categoiy, some programs rely on 
consciously worked out relationships between program managers or caseworkers and 
referral agencies to improve clients’ chances of getting needed services. These 
informal links are important because they are more common than formal SI 
arrangements. But because such networks may deteriorate when key staff leave, they 
cannot substitute for formal commitments in the long term except in rare instances in 
which an agency Is committed to holding the whole structure together (e.g.. Center for 
Family Life). We also found that the more formal SI arrangements have greater 
success than the informal ones in handling information sharing while respecting 
client confidentiality. 
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The programs with smooth-functioning SI arrangements cite several benefits, 
including: clients receive more and more appropriate services, participating agencies 
follow through on their commitments with greater speed and thoroughness, youth are 
much less likely to fall through the cracks, and staffing patterns stabilize because of 
the programs’ community-building philosophy (staff get excited to see how new 
mechanisms will work and want to be part of future developments). However, some 
difficulties typical of SI efforts were also encountered, including the usual difficulties 
with categorical funding and programs dependent on it, and turf issues between 
agencies and between professionals with different disciplinary training. At least one 
program also mentioned ethnic tensions in their larger service community over 
whether agencies affiliated with and serving particular ethnic groups would get their 
own resources or would have to refer to agencies affiliated with different ethnic groups 
to get services for their clients. 

Implications . Although some barriers to SI still need to be addressed, thre are 
some clear benefits for programs and their clients that make it worth the effort to 
undertake certain well-established SI activities (e.g., multi-agency teams), 

Conceptualizing SI More Broadly 

This project began with a view of service integration that is client-driven. It 
assumes that an agency has clients with service needs that it cannot meet entirely 
with its own resources, and that it becomes involved in formal interagency linkages to 
access services for its clients. We have learned that this view of SI is quite narrow 
and formal. It does not encompass several of the situations found during site visits, 
which appear to the researchers to epitomize an ideal of SI as service development 
and community coordination. Several programs we visited make themselves available 
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to develop services as their need is manifested in the community. If the program itself 
or other agencies with youth-serving responsibilities identify major unmet needs, the 
community of agencies can negotiate exactly what is needed, who can best provide it. 
how the various agencies in town will relate to the new service, and other similar 
issues. These agencies serve as mortar for their community networks--they hold them 
together, fill in the gaps, and facilitate smooth service delivery whether through their 
own services or the services of other agencies. They may do relatively little through 
formal or even informal referrals of their clients to other agencies, yet they help create 
a truly integrated service delivery system. 

Implications . The view of SI with which we began this study is too narrow, and 
does not capture many of the most creative activities of some of the programs we 
visited. It is too formal, and too driven by a standard case management model. The 
concept of SI should be expanded to accommodate the capacity-building activities of 
service development and community empowerment described in this report. 

Evaluation Issues 

Two key conditions determine a program’s readiness for outcome evaluation: 
willingness and capability. Among the nine programs visited, generally those with the 
highest levels of capability are also those with more positive attitudes toward 
evaluation, but this is not always true. 

In terms of willingness, most of the programs are interested in doing more 
evaluation research and assessing program outcomes in particular. Many directors 
specifically indicate that they want to do some form of longitudinal follow-up of their 
clients as an indicator of their program’s success. Enthusiasm tends to vary with the 
programs’ earlier positive experiences with evaluation, but also with the programs’ 



desire to document their impact. 

In terms of capability, the programs reveal low. moderate, and high capability. 
Low-capability programs lack the resources necessary for an evaluation study, 
including staff knowledgeable about evaluation research, computer capabilities, 
and a unit or department responsible for putting information together. These 
programs have some trouble tracking the involvement of outside agencies, which is an 
important component of documentation for SI types of programs. 

Moderate-capability programs maintain some form of computerized database 
system into which service and client statistics are entered regularly, and use their 
data for planning, internal evaluation, and reporting to funders. Such programs 
sometimes have quite specific plans for improving their evaluation capability. While 
these programs have strong interest in evaluation, some feel resources available for 
evaluations are insufficient. 

High-capability programs have highly sophisticated management information 
systems and staff specifically assigned to do the data entry, compilation, and 
summary statistics. They have usually conducted some form of evaluation in the past 
or currently. All of these programs clearly indicate that the benefits of the information 
obtained more than compensate for any costs incurred in doing evaluation research. 

Implications . The high-capability programs appear most ready to participate in 
a multi-program evaluation, and with some additional resources the moderate- 
capability programs may also move toward evaluation readiness. However, at least 
two issues need to be addressed in order to design an outcome evaluation that 
includes some or all of the sites visited and that will identify the effects of 
comprehensive service provision and SI models. The first issue concerns the choice of 
comparison or control groups. The second issue is how to resolve the variability in 
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information currently documented by programs, particularly client risk level 
information and service provision characteristics that would be amenable to 
classification. It would also be important to identify a standard minimum data set 
that all sites provide for the cross-site analysis. 

Evaluators should involve all programs in a cross-site evaluation decision- 
making about the evaluation design. Their involvement will improve the chance that 
the evaluation will reflect important program components in all programs, and will 
also help develop ownership and positive attitudes toward a cross-site evaluation. 

Given the special features of these programs, the measures should assess not only 
impacts on individual youth, but should also identify program effects on the 
community and on the service delivery network among community agencies. 

Conclusions 

This report has documented how nine programs deliver a wide variety of 
prevention and treatment services to at-risk youth between the ages of 10 and 15. It 
identifies a number of approaches to service delivery that appear to be very effective in 
increasing the comprehensiveness of service offerings and the potential impact on 
youth, their families, and their communities. These approaches often include an 
element of service Integration and one of our nine programs (CIS) is structured 
entirely to promote comprehensive service delivery through the service integration 
mechanism of co-location of services. These nine programs experience many of the 
common barriers to service integration, but still find that their clients derive some 
benefits from their SI efforts. Our findings also indicate that current conceptions of SI 
may be too narrow and formal. Our site visit experiences lead us to suggest a broader 
way of thinking about service integration that should be further explored in future 
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